[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:19 dist-test-slave-twwt krb5kdc[2473](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:19 dist-test-slave-twwt krb5kdc[2473](info): set up 2 sockets
Aug 11 14:35:19 dist-test-slave-twwt krb5kdc[2473](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 14:35:20.693096 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.478s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:35:20 dist-test-slave-twwt krb5kdc[2473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:35:20.835023 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:41343
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:36059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:41343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:20.926463 2501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:20.926782 2501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:20.926827 2501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:20.928045 2501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:20.928128 2501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:20.928151 2501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:20.928171 2501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:20.928203 2501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:20.930023 2501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:36059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:41343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:41343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:20.930498 2501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:20.930833 2501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:20.938066 2508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:20.938149 2509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:20.938196 2501 server_base.cc:1034] running on GCE node
W20260811 14:35:20.939685 2511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:20.940055 2501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:20.940492 2501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:20.941646 2501 hybrid_clock.cc:648] HybridClock initialized: now 1786458920941610 us; error 85 us; skew 500 ppm
Aug 11 14:35:20 dist-test-slave-twwt krb5kdc[2473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458920, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:20.944530 2501 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:35:20.945291 2501 webserver.cc:460] Webserver started at http://127.2.103.126:43013/ using document root <none> and password file <none>
I20260811 14:35:20.945624 2501 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:20.945708 2501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:20.945925 2501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:20.947396 2501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "2457edb03e0c4868a1f54fe376faffbe"
format_stamp: "Formatted at 2026-08-11 14:35:20 on dist-test-slave-twwt"
server_key: "02773e85e218a2879a05a68d8f90e82e"
server_key_iv: "cd29d4912941162c1e8f89132c8fe30a"
server_key_version: "encryptionkey@0"
I20260811 14:35:20.947880 2501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "2457edb03e0c4868a1f54fe376faffbe"
format_stamp: "Formatted at 2026-08-11 14:35:20 on dist-test-slave-twwt"
server_key: "02773e85e218a2879a05a68d8f90e82e"
server_key_iv: "cd29d4912941162c1e8f89132c8fe30a"
server_key_version: "encryptionkey@0"
I20260811 14:35:20.949955 2501 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 14:35:20.951210 2521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:20.951576 2501 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:35:20.951754 2501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "2457edb03e0c4868a1f54fe376faffbe"
format_stamp: "Formatted at 2026-08-11 14:35:20 on dist-test-slave-twwt"
server_key: "02773e85e218a2879a05a68d8f90e82e"
server_key_iv: "cd29d4912941162c1e8f89132c8fe30a"
server_key_version: "encryptionkey@0"
I20260811 14:35:20.951898 2501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:20.966986 2501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:20.969820 2501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:20.970052 2501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:20.975455 2501 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:41343
I20260811 14:35:20.975666 2611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:41343 every 8 connection(s)
I20260811 14:35:20.976205 2501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:20.977789 2612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:20.981333 2612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe: Bootstrap starting.
I20260811 14:35:20.982101 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 2501
I20260811 14:35:20.982245 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:35:20.982334 2612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:20.982810 2612 log.cc:824] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:20.983085 2461 external_mini_cluster.cc:1428] Setting key 285d14afc83288adb02f8ca7a5bac204
I20260811 14:35:20.983819 2612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe: No bootstrap required, opened a new log
I20260811 14:35:20.986555 2612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2457edb03e0c4868a1f54fe376faffbe" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41343 } }
I20260811 14:35:20.986827 2612 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:20.986939 2612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2457edb03e0c4868a1f54fe376faffbe, State: Initialized, Role: FOLLOWER
I20260811 14:35:20.987155 2612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2457edb03e0c4868a1f54fe376faffbe" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41343 } }
I20260811 14:35:20.987469 2612 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:20.987581 2612 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:20.987639 2612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:20.989048 2612 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2457edb03e0c4868a1f54fe376faffbe" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41343 } }
I20260811 14:35:20.989295 2612 leader_election.cc:302] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2457edb03e0c4868a1f54fe376faffbe; no voters:
I20260811 14:35:20.989559 2612 leader_election.cc:288] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:20.989593 2620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:20.989804 2620 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe [term 1 LEADER]: Becoming Leader. State: Replica: 2457edb03e0c4868a1f54fe376faffbe, State: Running, Role: LEADER
I20260811 14:35:20.990013 2620 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2457edb03e0c4868a1f54fe376faffbe" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41343 } }
I20260811 14:35:20.990073 2612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:20.990645 2622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2457edb03e0c4868a1f54fe376faffbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2457edb03e0c4868a1f54fe376faffbe" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41343 } } }
I20260811 14:35:20.990732 2622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:20.990629 2620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2457edb03e0c4868a1f54fe376faffbe. Latest consensus state: current_term: 1 leader_uuid: "2457edb03e0c4868a1f54fe376faffbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2457edb03e0c4868a1f54fe376faffbe" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41343 } } }
I20260811 14:35:20.991139 2620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:20.992048 2635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:20.993374 2635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:20.995646 2635 catalog_manager.cc:1347] Generated new cluster ID: eea7d2a6fff84ca5992a0a9e8c36d090
I20260811 14:35:20.995718 2635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:35:20 dist-test-slave-twwt krb5kdc[2473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:21.001905 2616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:20.984261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51060 (local address 127.2.103.126:41343)
0811 14:35:20.984536 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:20.984545 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:20.984768 (+ 223us) server_negotiation.cc:407] Connection header received
0811 14:35:20.987425 (+ 2657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:20.987431 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:20.987475 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:20.987644 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:20.989331 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:20.991005 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:20.992526 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:20.993350 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:20.993461 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:20.998035 (+ 4574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:20.998039 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:20.998040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:20.998052 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:20.999810 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:21.000289 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:21.000292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:21.000292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:21.000402 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:21.000685 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:21.000687 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:21.000687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:21.000946 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:21.001161 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:21.001446 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 14:35:21.001593 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":123,"threads_started":1}
I20260811 14:35:21.006454 2635 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:21.007131 2635 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:21.014283 2635 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2457edb03e0c4868a1f54fe376faffbe: Generated new TSK 0
I20260811 14:35:21.014508 2635 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
I20260811 14:35:21.047529 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41343
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:36059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:21.141964 2661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:21.142225 2661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:21.142267 2661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:21.143522 2661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:21.143596 2661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:21.143644 2661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:35:21.145227 2661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:36059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41343
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:21.145552 2661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:21.145812 2661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:21.152380 2672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:21.152493 2674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:21.152689 2661 server_base.cc:1034] running on GCE node
W20260811 14:35:21.152388 2670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:21.153065 2661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:21.153401 2661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:21.154563 2661 hybrid_clock.cc:648] HybridClock initialized: now 1786458921154548 us; error 33 us; skew 500 ppm
Aug 11 14:35:21 dist-test-slave-twwt krb5kdc[2473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458921, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:21.157706 2661 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:35:21.158294 2661 webserver.cc:460] Webserver started at http://127.2.103.65:34699/ using document root <none> and password file <none>
I20260811 14:35:21.158552 2661 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:21.158664 2661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:21.158854 2661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:21.160013 2661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "9347032615c847c882e4cf6d7bdcbf47"
format_stamp: "Formatted at 2026-08-11 14:35:21 on dist-test-slave-twwt"
server_key: "73a2fe33a7baac7e0519a3d7ab417345"
server_key_iv: "bb543f767c56aa90eaf5b8065e00d0e9"
server_key_version: "encryptionkey@0"
I20260811 14:35:21.160447 2661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "9347032615c847c882e4cf6d7bdcbf47"
format_stamp: "Formatted at 2026-08-11 14:35:21 on dist-test-slave-twwt"
server_key: "73a2fe33a7baac7e0519a3d7ab417345"
server_key_iv: "bb543f767c56aa90eaf5b8065e00d0e9"
server_key_version: "encryptionkey@0"
I20260811 14:35:21.161999 2661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:35:21.162813 2684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:21.163081 2661 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:35:21.163210 2661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "9347032615c847c882e4cf6d7bdcbf47"
format_stamp: "Formatted at 2026-08-11 14:35:21 on dist-test-slave-twwt"
server_key: "73a2fe33a7baac7e0519a3d7ab417345"
server_key_iv: "bb543f767c56aa90eaf5b8065e00d0e9"
server_key_version: "encryptionkey@0"
I20260811 14:35:21.163331 2661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:21.182901 2661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:21.185077 2661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:21.185199 2661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:21.185474 2661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:21.185819 2661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:21.185894 2661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:21.185920 2661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:21.185942 2661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:21.193930 2661 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:41485
I20260811 14:35:21.193996 2879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:41485 every 8 connection(s)
I20260811 14:35:21.194404 2661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:35:21.194908 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 2661
I20260811 14:35:21.195029 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:35:21.195212 2461 external_mini_cluster.cc:1428] Setting key 5988d4198d9086542f3389fd816b596f
Aug 11 14:35:21 dist-test-slave-twwt krb5kdc[2473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458921, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.103.66@KRBTEST.COM; defaulting to no policy
I20260811 14:35:21.207276 2616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:21.195445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:56255 (local address 127.2.103.126:41343)
0811 14:35:21.195507 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:21.195509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:21.195846 (+ 337us) server_negotiation.cc:407] Connection header received
0811 14:35:21.196371 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:21.196373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:21.196386 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:21.196453 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:21.197650 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:21.198262 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:21.199907 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:21.200106 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:21.200165 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:21.203398 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:21.203401 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:21.203402 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:21.203411 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:21.205700 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:21.206234 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:21.206235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:21.206236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:21.206295 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:21.206563 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:21.206563 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:21.206564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:21.206736 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:21.206793 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:21.207117 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 14:35:21.207205 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:35:21.207459 2884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:21.195452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:41343 (local address 127.2.103.65:56255)
0811 14:35:21.195635 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:21.195684 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:21.196258 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:21.196524 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:21.196527 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:21.197344 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:21.197545 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:21.197550 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:21.198381 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:21.198382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:21.199813 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:21.199817 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:21.200204 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:21.200205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:21.200355 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:21.201095 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:21.201107 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:21.203265 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:21.205825 (+ 2560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:21.205835 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:21.205838 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:21.206160 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:21.206384 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:21.206385 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:21.206385 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:21.206502 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:21.206822 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:21.206829 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:21.206982 (+ 153us) client_negotiation.cc:769] Sending connection context
0811 14:35:21.207077 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 14:35:21.207219 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
I20260811 14:35:21.208257 2880 heartbeater.cc:347] Connected to a master server at 127.2.103.126:41343
I20260811 14:35:21.208360 2880 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:21.208585 2880 heartbeater.cc:511] Master 127.2.103.126:41343 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.66@KRBTEST.COM" created.
I20260811 14:35:21.209687 2551 ts_manager.cc:194] Registered new tserver with Master: 9347032615c847c882e4cf6d7bdcbf47 (127.2.103.65:41485)
I20260811 14:35:21.210570 2551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:56255
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
I20260811 14:35:21.228386 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41343
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:36059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:21.315753 2890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:21.315979 2890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:21.316005 2890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:21.317226 2890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:21.317276 2890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:21.317305 2890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:35:21.318868 2890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:36059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41343
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:21.319139 2890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:21.319397 2890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:21.324978 2901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:21.325018 2902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:21.325012 2905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:21.325502 2890 server_base.cc:1034] running on GCE node
I20260811 14:35:21.325696 2890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:21.325937 2890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:21.327101 2890 hybrid_clock.cc:648] HybridClock initialized: now 1786458921327084 us; error 35 us; skew 500 ppm
Aug 11 14:35:21 dist-test-slave-twwt krb5kdc[2473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458921, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:21.329857 2890 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:35:21.330354 2890 webserver.cc:460] Webserver started at http://127.2.103.66:39893/ using document root <none> and password file <none>
I20260811 14:35:21.330547 2890 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:21.330590 2890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:21.330771 2890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:21.331815 2890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "b07ff065f07a4a84aac9b73abdfc4e1e"
format_stamp: "Formatted at 2026-08-11 14:35:21 on dist-test-slave-twwt"
server_key: "87c02cc93e878a0277ba433c535287d1"
server_key_iv: "79e6b00a71c94fbacd7edf58f2d9cbd5"
server_key_version: "encryptionkey@0"
I20260811 14:35:21.332219 2890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "b07ff065f07a4a84aac9b73abdfc4e1e"
format_stamp: "Formatted at 2026-08-11 14:35:21 on dist-test-slave-twwt"
server_key: "87c02cc93e878a0277ba433c535287d1"
server_key_iv: "79e6b00a71c94fbacd7edf58f2d9cbd5"
server_key_version: "encryptionkey@0"
I20260811 14:35:21.333767 2890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:21.334733 2915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:21.334954 2890 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:21.335071 2890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "b07ff065f07a4a84aac9b73abdfc4e1e"
format_stamp: "Formatted at 2026-08-11 14:35:21 on dist-test-slave-twwt"
server_key: "87c02cc93e878a0277ba433c535287d1"
server_key_iv: "79e6b00a71c94fbacd7edf58f2d9cbd5"
server_key_version: "encryptionkey@0"
I20260811 14:35:21.335170 2890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:21.360292 2890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:21.362968 2890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:21.363154 2890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:21.363431 2890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:21.363797 2890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:21.363862 2890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:21.363960 2890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:21.363993 2890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:21.372414 2890 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:38877
I20260811 14:35:21.372470 3118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:38877 every 8 connection(s)
I20260811 14:35:21.372880 2890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:35:21.374943 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 2890
I20260811 14:35:21.375033 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:35:21.375221 2461 external_mini_cluster.cc:1428] Setting key adea06e314ada0285d9069167978adfb
Aug 11 14:35:21 dist-test-slave-twwt krb5kdc[2473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458921, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:21.384231 2616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:21.373742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:58657 (local address 127.2.103.126:41343)
0811 14:35:21.373827 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:21.373830 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:21.374131 (+ 301us) server_negotiation.cc:407] Connection header received
0811 14:35:21.374650 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:21.374652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:21.374665 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:21.374765 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:21.376030 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:21.376826 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:21.377918 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:21.378257 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:21.378339 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:21.380857 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:21.380859 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:21.380859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:21.380869 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:21.382692 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:21.383114 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:21.383115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:21.383116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:21.383188 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:21.383462 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:21.383463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:21.383464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:21.383615 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:21.383700 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:21.384009 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 14:35:21.384120 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 14:35:21.384266 3126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:21.373732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:41343 (local address 127.2.103.66:58657)
0811 14:35:21.373971 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:21.374010 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:21.374501 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:21.374801 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:21.374805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:21.375622 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:21.375922 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:21.375929 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:21.376956 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:21.376957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:21.377815 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:21.377817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:21.378361 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:21.378362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:21.378523 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:21.379101 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:21.379113 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:21.380745 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:21.382787 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:21.382794 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:21.382795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:21.383026 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:21.383273 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:21.383274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:21.383274 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:21.383383 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:21.383720 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:21.383726 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:21.383856 (+ 130us) client_negotiation.cc:769] Sending connection context
0811 14:35:21.383956 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 14:35:21.384092 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":153,"threads_started":1}
I20260811 14:35:21.384915 3119 heartbeater.cc:347] Connected to a master server at 127.2.103.126:41343
I20260811 14:35:21.385046 3119 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:21.385248 3119 heartbeater.cc:511] Master 127.2.103.126:41343 requested a full tablet report, sending...
I20260811 14:35:21.385694 2551 ts_manager.cc:194] Registered new tserver with Master: b07ff065f07a4a84aac9b73abdfc4e1e (127.2.103.66:38877)
WARNING: no policy specified for kudu/127.2.103.67@KRBTEST.COM; defaulting to no policy
I20260811 14:35:21.386113 2551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:58657
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
I20260811 14:35:21.406248 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41343
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:36059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:21.494174 3131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:21.494431 3131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:21.494475 3131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:21.495713 3131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:21.495788 3131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:21.495837 3131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:35:21.497335 3131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:36059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41343
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:21.497615 3131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:21.497915 3131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:21.503374 3146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:21.503408 3143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:21.503337 3142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:21.503762 3131 server_base.cc:1034] running on GCE node
I20260811 14:35:21.503978 3131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:21.504184 3131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:21.505366 3131 hybrid_clock.cc:648] HybridClock initialized: now 1786458921505349 us; error 42 us; skew 500 ppm
Aug 11 14:35:21 dist-test-slave-twwt krb5kdc[2473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458921, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:21.508059 3131 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:35:21.508538 3131 webserver.cc:460] Webserver started at http://127.2.103.67:37407/ using document root <none> and password file <none>
I20260811 14:35:21.508831 3131 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:21.508909 3131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:21.509084 3131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:21.510245 3131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "b7c7699c906b47c6857d54c1cf8f8e3d"
format_stamp: "Formatted at 2026-08-11 14:35:21 on dist-test-slave-twwt"
server_key: "f81ad548cbb9445b68c3e90643581898"
server_key_iv: "9e16267d8d27d13d9fd1ce46dc898b93"
server_key_version: "encryptionkey@0"
I20260811 14:35:21.510747 3131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "b7c7699c906b47c6857d54c1cf8f8e3d"
format_stamp: "Formatted at 2026-08-11 14:35:21 on dist-test-slave-twwt"
server_key: "f81ad548cbb9445b68c3e90643581898"
server_key_iv: "9e16267d8d27d13d9fd1ce46dc898b93"
server_key_version: "encryptionkey@0"
I20260811 14:35:21.512177 3131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:21.513051 3158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:21.513260 3131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:21.513402 3131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "b7c7699c906b47c6857d54c1cf8f8e3d"
format_stamp: "Formatted at 2026-08-11 14:35:21 on dist-test-slave-twwt"
server_key: "f81ad548cbb9445b68c3e90643581898"
server_key_iv: "9e16267d8d27d13d9fd1ce46dc898b93"
server_key_version: "encryptionkey@0"
I20260811 14:35:21.513494 3131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:21.535539 3131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:21.537748 3131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:21.537946 3131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:21.538247 3131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:21.538671 3131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:21.538730 3131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:21.538775 3131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:21.538803 3131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:21.548156 3131 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:46807
I20260811 14:35:21.548203 3355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:46807 every 8 connection(s)
I20260811 14:35:21.548578 3131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:35:21.553030 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 3131
I20260811 14:35:21.553107 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:35:21.553277 2461 external_mini_cluster.cc:1428] Setting key d230ff62e1936e7142e9c32c697232b2
Aug 11 14:35:21 dist-test-slave-twwt krb5kdc[2473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458921, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:21.558755 2616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:21.549491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:46049 (local address 127.2.103.126:41343)
0811 14:35:21.549578 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:21.549580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:21.549758 (+ 178us) server_negotiation.cc:407] Connection header received
0811 14:35:21.550249 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:21.550250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:21.550261 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:21.550329 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:21.551371 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:21.551872 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:21.552930 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:21.553154 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:21.553250 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:21.555984 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:21.555986 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:21.555986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:21.555995 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:21.557260 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:21.557690 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:21.557691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:21.557692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:21.557758 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:21.558034 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:21.558035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:21.558035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:21.558156 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:21.558238 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:21.558533 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 14:35:21.558665 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 14:35:21.558936 3360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:21.549438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:41343 (local address 127.2.103.67:46049)
0811 14:35:21.549631 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:21.549670 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:21.550163 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:21.550392 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:21.550396 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:21.551078 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:21.551282 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:21.551288 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:21.551983 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:21.551984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:21.552808 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:21.552811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:21.553260 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:21.553261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:21.553435 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:21.554124 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:21.554134 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:21.555886 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:21.557372 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:21.557379 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:21.557381 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:21.557608 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:21.557837 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:21.557838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:21.557839 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:21.557957 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:21.558229 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:21.558236 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:21.558388 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 14:35:21.558497 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 14:35:21.558690 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":103,"threads_started":1}
I20260811 14:35:21.559422 3358 heartbeater.cc:347] Connected to a master server at 127.2.103.126:41343
I20260811 14:35:21.559522 3358 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:21.559679 3358 heartbeater.cc:511] Master 127.2.103.126:41343 requested a full tablet report, sending...
I20260811 14:35:21.560056 2551 ts_manager.cc:194] Registered new tserver with Master: b7c7699c906b47c6857d54c1cf8f8e3d (127.2.103.67:46807)
I20260811 14:35:21.560447 2551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:46049
I20260811 14:35:21.565659 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 14:35:21 dist-test-slave-twwt krb5kdc[2473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458921, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 14:35:21 dist-test-slave-twwt krb5kdc[2473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458921, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.103.65@KRBTEST.COM
I20260811 14:35:21.594738 3370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:21.586107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41408 (local address 127.2.103.65:41485)
0811 14:35:21.586298 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:21.586308 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:21.586323 (+ 15us) server_negotiation.cc:407] Connection header received
0811 14:35:21.586394 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:21.586397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:21.586412 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:21.586673 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:21.587285 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:21.588047 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:21.588761 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:21.589078 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:21.589134 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:21.591631 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:21.591640 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:21.591640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:21.591647 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:21.593359 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:21.593756 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:21.593760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:21.593760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:21.593831 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:21.594134 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:21.594135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:21.594135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:21.594284 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:21.594421 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:21.594520 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 14:35:21.594593 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":112,"threads_started":1}
W20260811 14:35:21.595328 2725 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41408
Aug 11 14:35:21 dist-test-slave-twwt krb5kdc[2473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 14:35:21 dist-test-slave-twwt krb5kdc[2473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.65@KRBTEST.COM
I20260811 14:35:21.624584 3370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:21.616930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41418 (local address 127.2.103.65:41485)
0811 14:35:21.617014 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:21.617016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:21.617138 (+ 122us) server_negotiation.cc:407] Connection header received
0811 14:35:21.617228 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:21.617229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:21.617245 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:21.617314 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:21.618002 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:21.618546 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:21.619340 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:21.619554 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:21.619640 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:21.621987 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:21.621989 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:21.621989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:21.621997 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:21.623346 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:21.623754 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:21.623755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:21.623756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:21.623813 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:21.624039 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:21.624040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:21.624040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:21.624148 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:21.624273 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:21.624379 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 14:35:21.624473 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 14:35:21.625204 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 2661
I20260811 14:35:21.643728 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 2890
I20260811 14:35:21.654498 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 3131
I20260811 14:35:21.682971 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 2501
2026-08-11T14:35:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2512 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:21 dist-test-slave-twwt krb5kdc[3380](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:21 dist-test-slave-twwt krb5kdc[3380](info): set up 2 sockets
Aug 11 14:35:21 dist-test-slave-twwt krb5kdc[3380](info): commencing operation
krb5kdc: starting...
W20260811 14:35:23.034714 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.314s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:35:23 dist-test-slave-twwt krb5kdc[3380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:35:23.170740 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:41121
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:34585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:41121
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:23.259187 3402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:23.259424 3402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:23.259449 3402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:23.260557 3402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:23.260593 3402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:23.260607 3402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:23.260620 3402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:23.260632 3402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:23.262064 3402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:34585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:41121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:41121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:23.262269 3402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:23.262475 3402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:23.267632 3418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:23.267668 3412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:23.267668 3413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:23.267954 3402 server_base.cc:1034] running on GCE node
I20260811 14:35:23.268224 3402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:23.268545 3402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:23.269716 3402 hybrid_clock.cc:648] HybridClock initialized: now 1786458923269694 us; error 40 us; skew 500 ppm
Aug 11 14:35:23 dist-test-slave-twwt krb5kdc[3380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458923, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:23.278388 3402 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:35:23.278954 3402 webserver.cc:460] Webserver started at http://127.2.103.126:39991/ using document root <none> and password file <none>
I20260811 14:35:23.279210 3402 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:23.279289 3402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:23.279444 3402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:23.280575 3402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "84d6ea20cc8f4f068458192530ae1e68"
format_stamp: "Formatted at 2026-08-11 14:35:23 on dist-test-slave-twwt"
server_key: "d29667ce46c22a1378a6a777288e2526"
server_key_iv: "e06f377d3ee326d280eff27c8b4c5a39"
server_key_version: "encryptionkey@0"
I20260811 14:35:23.281009 3402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "84d6ea20cc8f4f068458192530ae1e68"
format_stamp: "Formatted at 2026-08-11 14:35:23 on dist-test-slave-twwt"
server_key: "d29667ce46c22a1378a6a777288e2526"
server_key_iv: "e06f377d3ee326d280eff27c8b4c5a39"
server_key_version: "encryptionkey@0"
I20260811 14:35:23.282509 3402 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:23.283425 3426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:23.283622 3402 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:23.283735 3402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "84d6ea20cc8f4f068458192530ae1e68"
format_stamp: "Formatted at 2026-08-11 14:35:23 on dist-test-slave-twwt"
server_key: "d29667ce46c22a1378a6a777288e2526"
server_key_iv: "e06f377d3ee326d280eff27c8b4c5a39"
server_key_version: "encryptionkey@0"
I20260811 14:35:23.283851 3402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:23.305678 3402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:23.308040 3402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:23.308228 3402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:23.312489 3402 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:41121
I20260811 14:35:23.312531 3505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:41121 every 8 connection(s)
I20260811 14:35:23.312928 3402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:23.313890 3510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:23.316365 3510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68: Bootstrap starting.
I20260811 14:35:23.316954 3510 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:23.317283 3510 log.cc:824] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:23.317312 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 3402
I20260811 14:35:23.317410 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:35:23.317595 2461 external_mini_cluster.cc:1428] Setting key f8bc4de46ce80039528c8d5d02a40f0c
I20260811 14:35:23.318259 3510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68: No bootstrap required, opened a new log
I20260811 14:35:23.320056 3510 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d6ea20cc8f4f068458192530ae1e68" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41121 } }
I20260811 14:35:23.320235 3510 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:23.320304 3510 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d6ea20cc8f4f068458192530ae1e68, State: Initialized, Role: FOLLOWER
I20260811 14:35:23.320465 3510 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d6ea20cc8f4f068458192530ae1e68" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41121 } }
I20260811 14:35:23.320572 3510 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:23.320621 3510 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:23.320672 3510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:23.321969 3510 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d6ea20cc8f4f068458192530ae1e68" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41121 } }
I20260811 14:35:23.322098 3510 leader_election.cc:302] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84d6ea20cc8f4f068458192530ae1e68; no voters:
I20260811 14:35:23.322311 3510 leader_election.cc:288] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:23.322347 3522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:23.322463 3522 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68 [term 1 LEADER]: Becoming Leader. State: Replica: 84d6ea20cc8f4f068458192530ae1e68, State: Running, Role: LEADER
I20260811 14:35:23.322625 3510 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:23.322651 3522 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d6ea20cc8f4f068458192530ae1e68" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41121 } }
I20260811 14:35:23.322992 3527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84d6ea20cc8f4f068458192530ae1e68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d6ea20cc8f4f068458192530ae1e68" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41121 } } }
I20260811 14:35:23.322975 3522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84d6ea20cc8f4f068458192530ae1e68. Latest consensus state: current_term: 1 leader_uuid: "84d6ea20cc8f4f068458192530ae1e68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d6ea20cc8f4f068458192530ae1e68" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41121 } } }
I20260811 14:35:23.323210 3522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:23.323447 3527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:23.323463 3537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:23.324120 3537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:23.326275 3537 catalog_manager.cc:1347] Generated new cluster ID: f9bfcd74340941a685d4e5d6ab7e2230
I20260811 14:35:23.326351 3537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:35:23 dist-test-slave-twwt krb5kdc[3380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:23.332870 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:23.318229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41954 (local address 127.2.103.126:41121)
0811 14:35:23.318461 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:23.318469 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:23.318496 (+ 27us) server_negotiation.cc:407] Connection header received
0811 14:35:23.320066 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:23.320076 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:23.320141 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:23.320356 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:23.321222 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.322703 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.323941 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.324788 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.324936 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:23.328049 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:23.328055 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:23.328055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:23.328071 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:23.330085 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:23.330551 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:23.330555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:23.330556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:23.331511 (+ 955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:23.331775 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:23.331776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:23.331777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:23.332117 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:23.332306 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:23.332444 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 14:35:23.332616 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":92,"threads_started":1}
I20260811 14:35:23.340979 3537 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:23.341565 3537 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:23.343791 3537 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84d6ea20cc8f4f068458192530ae1e68: Generated new TSK 0
I20260811 14:35:23.343997 3537 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
I20260811 14:35:23.378254 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41121
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:34585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:23.471616 3560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:23.471843 3560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:23.471868 3560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:23.473076 3560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:23.473129 3560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:23.473157 3560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:35:23.474856 3560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:34585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41121
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:23.475129 3560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:23.475383 3560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:23.480768 3572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:23.480857 3560 server_base.cc:1034] running on GCE node
W20260811 14:35:23.480809 3568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:23.480768 3567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:23.481272 3560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:23.481556 3560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:23.482697 3560 hybrid_clock.cc:648] HybridClock initialized: now 1786458923482678 us; error 33 us; skew 500 ppm
Aug 11 14:35:23 dist-test-slave-twwt krb5kdc[3380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458923, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:23.485486 3560 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:35:23.485999 3560 webserver.cc:460] Webserver started at http://127.2.103.65:38215/ using document root <none> and password file <none>
I20260811 14:35:23.486284 3560 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:23.486335 3560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:23.486477 3560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:23.487660 3560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "12aef800531d4b458021b3fea32d664f"
format_stamp: "Formatted at 2026-08-11 14:35:23 on dist-test-slave-twwt"
server_key: "7fd0c6dd1936cec9f699e8f8e9f1eec8"
server_key_iv: "a7fb08234259e83aada7dc2e6e623178"
server_key_version: "encryptionkey@0"
I20260811 14:35:23.488082 3560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "12aef800531d4b458021b3fea32d664f"
format_stamp: "Formatted at 2026-08-11 14:35:23 on dist-test-slave-twwt"
server_key: "7fd0c6dd1936cec9f699e8f8e9f1eec8"
server_key_iv: "a7fb08234259e83aada7dc2e6e623178"
server_key_version: "encryptionkey@0"
I20260811 14:35:23.489467 3560 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:23.490356 3584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:23.490595 3560 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:23.490764 3560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "12aef800531d4b458021b3fea32d664f"
format_stamp: "Formatted at 2026-08-11 14:35:23 on dist-test-slave-twwt"
server_key: "7fd0c6dd1936cec9f699e8f8e9f1eec8"
server_key_iv: "a7fb08234259e83aada7dc2e6e623178"
server_key_version: "encryptionkey@0"
I20260811 14:35:23.490865 3560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:23.517066 3560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:23.519757 3560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:23.519960 3560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:23.520231 3560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:23.520589 3560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:23.520648 3560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:23.520691 3560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:23.520714 3560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:23.527529 3560 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:43607
I20260811 14:35:23.527588 3766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:43607 every 8 connection(s)
I20260811 14:35:23.527922 3560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:35:23 dist-test-slave-twwt krb5kdc[3380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458923, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:23.535173 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 3560
I20260811 14:35:23.535293 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:35:23.535504 2461 external_mini_cluster.cc:1428] Setting key 55faecf7331ce4e3dcb3c2d2c3dbc4e2
I20260811 14:35:23.537878 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:23.528659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:34809 (local address 127.2.103.126:41121)
0811 14:35:23.528728 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:23.528729 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:23.528985 (+ 256us) server_negotiation.cc:407] Connection header received
0811 14:35:23.530234 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:23.530235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:23.530252 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:23.530316 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:23.530953 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.531470 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.532462 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.532660 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.532726 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:23.534834 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:23.534835 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:23.534835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:23.534844 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:23.536454 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:23.536864 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:23.536865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:23.536866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:23.536932 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:23.537168 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:23.537169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:23.537169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:23.537316 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:23.537388 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:23.537626 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 14:35:23.537724 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 14:35:23.537902 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:23.528639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:41121 (local address 127.2.103.65:34809)
0811 14:35:23.528821 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:23.528898 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:23.530121 (+ 1223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:23.530379 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:23.530383 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:23.530661 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:23.530863 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:23.530867 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.531555 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.531556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:23.532388 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:23.532391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.532750 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.532751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:23.532877 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:23.533349 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:23.533358 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:23.534751 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:23.536547 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:23.536553 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:23.536555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:23.536782 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:23.537000 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:23.537001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:23.537001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:23.537101 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:23.537382 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:23.537386 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:23.537495 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 14:35:23.537579 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 14:35:23.537697 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
I20260811 14:35:23.538579 3767 heartbeater.cc:347] Connected to a master server at 127.2.103.126:41121
I20260811 14:35:23.538733 3767 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:23.538944 3767 heartbeater.cc:511] Master 127.2.103.126:41121 requested a full tablet report, sending...
I20260811 14:35:23.539563 3453 ts_manager.cc:194] Registered new tserver with Master: 12aef800531d4b458021b3fea32d664f (127.2.103.65:43607)
I20260811 14:35:23.540227 3453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:34809
WARNING: no policy specified for kudu/127.2.103.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
I20260811 14:35:23.565860 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41121
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:34585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:23.652611 3776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:23.652971 3776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:23.653061 3776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:23.654342 3776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:23.654387 3776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:23.654414 3776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:35:23.655989 3776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:34585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41121
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:23.656255 3776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:23.656518 3776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:23.661722 3790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:23.661749 3786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:23.661744 3788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:23.661981 3776 server_base.cc:1034] running on GCE node
I20260811 14:35:23.662197 3776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:23.662405 3776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:23.663546 3776 hybrid_clock.cc:648] HybridClock initialized: now 1786458923663508 us; error 58 us; skew 500 ppm
Aug 11 14:35:23 dist-test-slave-twwt krb5kdc[3380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458923, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:23.666227 3776 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:35:23.666750 3776 webserver.cc:460] Webserver started at http://127.2.103.66:34361/ using document root <none> and password file <none>
I20260811 14:35:23.667016 3776 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:23.667090 3776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:23.667249 3776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:23.668318 3776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "f6a8b39849444152a003f5b71e7aee42"
format_stamp: "Formatted at 2026-08-11 14:35:23 on dist-test-slave-twwt"
server_key: "4830aa5efb2afa9c0f56006c29e7bac5"
server_key_iv: "ddb2344cd8d985a697175def6a0d15b4"
server_key_version: "encryptionkey@0"
I20260811 14:35:23.668713 3776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "f6a8b39849444152a003f5b71e7aee42"
format_stamp: "Formatted at 2026-08-11 14:35:23 on dist-test-slave-twwt"
server_key: "4830aa5efb2afa9c0f56006c29e7bac5"
server_key_iv: "ddb2344cd8d985a697175def6a0d15b4"
server_key_version: "encryptionkey@0"
I20260811 14:35:23.670109 3776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:35:23.671010 3801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:23.671213 3776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:35:23.671336 3776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "f6a8b39849444152a003f5b71e7aee42"
format_stamp: "Formatted at 2026-08-11 14:35:23 on dist-test-slave-twwt"
server_key: "4830aa5efb2afa9c0f56006c29e7bac5"
server_key_iv: "ddb2344cd8d985a697175def6a0d15b4"
server_key_version: "encryptionkey@0"
I20260811 14:35:23.671430 3776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:23.702257 3776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:23.704844 3776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:23.704967 3776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:23.705178 3776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:23.705535 3776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:23.705572 3776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:23.705596 3776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:23.705646 3776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:23.711720 3776 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:38515
I20260811 14:35:23.711817 3998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:38515 every 8 connection(s)
I20260811 14:35:23.712169 3776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
Aug 11 14:35:23 dist-test-slave-twwt krb5kdc[3380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458923, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:23.721658 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:23.713045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:46879 (local address 127.2.103.126:41121)
0811 14:35:23.713125 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:23.713128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:23.713354 (+ 226us) server_negotiation.cc:407] Connection header received
0811 14:35:23.714370 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:23.714371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:23.714383 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:23.714444 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:23.715095 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.715603 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.716633 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.716834 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.716897 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:23.719149 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:23.719150 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:23.719151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:23.719157 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:23.720385 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:23.720864 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:23.720864 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:23.720865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:23.720917 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:23.721136 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:23.721136 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:23.721137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:23.721237 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:23.721286 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:23.721520 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 14:35:23.721589 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 14:35:23.721905 4003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:23.713018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:41121 (local address 127.2.103.66:46879)
0811 14:35:23.713211 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:23.713255 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:23.714241 (+ 986us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:23.714501 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:23.714505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:23.714791 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:23.715003 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:23.715008 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.715695 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.715696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:23.716542 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:23.716545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.716921 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.716922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:23.717024 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:23.717593 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:23.717602 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:23.719057 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:23.720461 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:23.720467 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:23.720470 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:23.720780 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:23.720975 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:23.720975 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:23.720976 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:23.721079 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:23.721291 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:23.721295 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:23.721406 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 14:35:23.721494 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 14:35:23.721615 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":101,"threads_started":1}
I20260811 14:35:23.722316 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 3776
I20260811 14:35:23.722425 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:35:23.722533 4001 heartbeater.cc:347] Connected to a master server at 127.2.103.126:41121
I20260811 14:35:23.722651 2461 external_mini_cluster.cc:1428] Setting key 621a8074d100d0b6257c2a4603cd90ef
I20260811 14:35:23.722661 4001 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:23.722873 4001 heartbeater.cc:511] Master 127.2.103.126:41121 requested a full tablet report, sending...
I20260811 14:35:23.723337 3453 ts_manager.cc:194] Registered new tserver with Master: f6a8b39849444152a003f5b71e7aee42 (127.2.103.66:38515)
I20260811 14:35:23.723706 3453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:46879
WARNING: no policy specified for kudu/127.2.103.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
I20260811 14:35:23.751278 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41121
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:34585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:23.838217 4006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:23.838467 4006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:23.838510 4006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:23.839735 4006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:23.839811 4006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:23.839861 4006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:35:23.841389 4006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:34585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41121
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:23.841663 4006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:23.841933 4006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:23.847237 4015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:23.847254 4022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:23.847247 4016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:23.847506 4006 server_base.cc:1034] running on GCE node
I20260811 14:35:23.847810 4006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:23.848091 4006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:23.849265 4006 hybrid_clock.cc:648] HybridClock initialized: now 1786458923849235 us; error 64 us; skew 500 ppm
Aug 11 14:35:23 dist-test-slave-twwt krb5kdc[3380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458923, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:23.851977 4006 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:35:23.852386 4006 webserver.cc:460] Webserver started at http://127.2.103.67:46321/ using document root <none> and password file <none>
I20260811 14:35:23.852596 4006 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:23.852639 4006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:23.852747 4006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:23.853772 4006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "1c0ec704dd254432a62e274805280548"
format_stamp: "Formatted at 2026-08-11 14:35:23 on dist-test-slave-twwt"
server_key: "0b3d8dfe31fc5721ff533b827f348edf"
server_key_iv: "6dad9bf28cbf7454979366644f2502f5"
server_key_version: "encryptionkey@0"
I20260811 14:35:23.854092 4006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "1c0ec704dd254432a62e274805280548"
format_stamp: "Formatted at 2026-08-11 14:35:23 on dist-test-slave-twwt"
server_key: "0b3d8dfe31fc5721ff533b827f348edf"
server_key_iv: "6dad9bf28cbf7454979366644f2502f5"
server_key_version: "encryptionkey@0"
I20260811 14:35:23.855540 4006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260811 14:35:23.856212 4031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:23.856419 4006 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:23.856504 4006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "1c0ec704dd254432a62e274805280548"
format_stamp: "Formatted at 2026-08-11 14:35:23 on dist-test-slave-twwt"
server_key: "0b3d8dfe31fc5721ff533b827f348edf"
server_key_iv: "6dad9bf28cbf7454979366644f2502f5"
server_key_version: "encryptionkey@0"
I20260811 14:35:23.856559 4006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:23.874676 4006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:23.877316 4006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:23.877439 4006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:23.877635 4006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:23.877974 4006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:23.878010 4006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:23.878033 4006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:23.878082 4006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:23.885450 4006 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:35503
I20260811 14:35:23.885501 4210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:35503 every 8 connection(s)
I20260811 14:35:23.885872 4006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:35:23.887245 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 4006
I20260811 14:35:23.887374 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:35:23.887537 2461 external_mini_cluster.cc:1428] Setting key 2117a7d41bd67d0bd57911a8551ea4f5
Aug 11 14:35:23 dist-test-slave-twwt krb5kdc[3380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458923, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:23.896521 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:23.886878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:49493 (local address 127.2.103.126:41121)
0811 14:35:23.886925 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:23.886928 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:23.887225 (+ 297us) server_negotiation.cc:407] Connection header received
0811 14:35:23.888527 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:23.888528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:23.888541 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:23.888594 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:23.889281 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.889794 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.890901 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.891106 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.891192 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:23.893498 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:23.893500 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:23.893500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:23.893509 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:23.895014 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:23.895457 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:23.895458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:23.895458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:23.895519 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:23.895817 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:23.895818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:23.895818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:23.895945 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:23.896028 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:23.896336 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 14:35:23.896425 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:35:23.896714 4217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:23.886898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:41121 (local address 127.2.103.67:49493)
0811 14:35:23.887068 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:23.887106 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:23.888420 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:23.888683 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:23.888688 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:23.888961 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:23.889161 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:23.889165 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.889897 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.889898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:23.890813 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:23.890815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.891198 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.891200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:23.891336 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:23.891821 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:23.891829 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:23.893391 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:23.895112 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:23.895119 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:23.895122 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:23.895361 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:23.895589 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:23.895590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:23.895590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:23.895714 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:23.896019 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:23.896026 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:23.896175 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 14:35:23.896306 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 14:35:23.896475 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260811 14:35:23.897233 4212 heartbeater.cc:347] Connected to a master server at 127.2.103.126:41121
I20260811 14:35:23.897347 4212 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:23.897555 4212 heartbeater.cc:511] Master 127.2.103.126:41121 requested a full tablet report, sending...
I20260811 14:35:23.898018 3453 ts_manager.cc:194] Registered new tserver with Master: 1c0ec704dd254432a62e274805280548 (127.2.103.67:35503)
I20260811 14:35:23.898414 3453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:49493
I20260811 14:35:23.899461 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:35:23.907398 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:23.901429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41968 (local address 127.2.103.126:41121)
0811 14:35:23.901470 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:23.901472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:23.901575 (+ 103us) server_negotiation.cc:407] Connection header received
0811 14:35:23.901615 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:23.901616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:23.901624 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:23.901684 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:23.902200 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.902739 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.903458 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.903635 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.903685 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:23.904968 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:23.904970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:23.904970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:23.904976 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:23.906220 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:23.906573 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:23.906574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:23.906574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:23.906711 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:23.906937 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:23.906938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:23.906938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:23.907090 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:23.907169 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:23.907269 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 14:35:23.907333 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:35:23.910452 3453 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:35:23.912420 3453 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:35:23.920429 4235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:23.915937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:38515 (local address 127.0.0.1:53364)
0811 14:35:23.916326 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:23.916366 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:23.916550 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:23.917009 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:23.917010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:23.917013 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:23.917065 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:23.917067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.917987 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.917988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:23.919244 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:23.919247 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.920061 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.920062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:23.920159 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:23.920174 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 14:35:23.920284 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 14:35:23.920349 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":65,"threads_started":1}
I20260811 14:35:23.920439 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:23.916058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53364 (local address 127.2.103.66:38515)
0811 14:35:23.916698 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:23.916702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:23.916711 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:35:23.916757 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:23.916759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:23.916775 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:23.916893 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:23.917152 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.917876 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.919342 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.919957 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.920048 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:23.920117 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:23.920306 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 14:35:23.920349 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":95,"threads_started":1}
I20260811 14:35:23.920789 4243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:23.915942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:35503 (local address 127.0.0.1:46104)
0811 14:35:23.916403 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:23.916416 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:23.916524 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:23.916923 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:23.916926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:23.916928 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:23.917021 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:23.917023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.918383 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.918384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:23.919437 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:23.919439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.920494 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.920495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:23.920617 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:23.920624 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 14:35:23.920679 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 14:35:23.920720 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":3,"thread_start_us":204,"threads_started":1}
I20260811 14:35:23.920851 4242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:23.916119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46104 (local address 127.2.103.67:35503)
0811 14:35:23.916292 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:23.916297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:23.916527 (+ 230us) server_negotiation.cc:407] Connection header received
0811 14:35:23.916653 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:23.916657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:23.916677 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:23.916789 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:23.917117 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.918261 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.919551 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.920324 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.920399 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:23.920454 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:23.920722 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 14:35:23.920774 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
I20260811 14:35:23.921360 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:23.916151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:43607 (local address 127.0.0.1:49930)
0811 14:35:23.916643 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:23.916653 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:23.916743 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:23.917112 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:23.917113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:23.917116 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:23.917156 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:23.917157 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.918426 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.918427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:23.919515 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:23.919518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.920833 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.920834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:23.921162 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:23.921166 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:35:23.921247 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 14:35:23.921287 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"mutex_wait_us":2,"thread_start_us":67,"threads_started":1}
I20260811 14:35:23.921396 4244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:23.916396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49930 (local address 127.2.103.65:43607)
0811 14:35:23.916633 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:23.916638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:23.916883 (+ 245us) server_negotiation.cc:407] Connection header received
0811 14:35:23.916930 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:23.916932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:23.916946 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:23.917020 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:23.917243 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.918303 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.919612 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:23.920331 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:23.920399 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:23.920455 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:23.921268 (+ 813us) server_negotiation.cc:299] Negotiation successful
0811 14:35:23.921316 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":91,"threads_started":1}
I20260811 14:35:23.921379 3884 tablet_service.cc:1467] Processing CreateTablet for tablet e6f97aee9ce341aeb0cb64665441253f (DEFAULT_TABLE table=test-table [id=bdced8b277b74b708853171197bfa8a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:23.921494 4114 tablet_service.cc:1467] Processing CreateTablet for tablet e6f97aee9ce341aeb0cb64665441253f (DEFAULT_TABLE table=test-table [id=bdced8b277b74b708853171197bfa8a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:23.921808 4114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6f97aee9ce341aeb0cb64665441253f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:23.921808 3884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6f97aee9ce341aeb0cb64665441253f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:23.922348 3658 tablet_service.cc:1467] Processing CreateTablet for tablet e6f97aee9ce341aeb0cb64665441253f (DEFAULT_TABLE table=test-table [id=bdced8b277b74b708853171197bfa8a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:23.922720 3658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6f97aee9ce341aeb0cb64665441253f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:23.925681 4245 tablet_bootstrap.cc:492] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f: Bootstrap starting.
I20260811 14:35:23.925937 4249 tablet_bootstrap.cc:492] T e6f97aee9ce341aeb0cb64665441253f P f6a8b39849444152a003f5b71e7aee42: Bootstrap starting.
I20260811 14:35:23.926465 4249 tablet_bootstrap.cc:654] T e6f97aee9ce341aeb0cb64665441253f P f6a8b39849444152a003f5b71e7aee42: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:23.926472 4245 tablet_bootstrap.cc:654] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:23.927142 4249 log.cc:824] T e6f97aee9ce341aeb0cb64665441253f P f6a8b39849444152a003f5b71e7aee42: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:23.927691 4247 tablet_bootstrap.cc:492] T e6f97aee9ce341aeb0cb64665441253f P 1c0ec704dd254432a62e274805280548: Bootstrap starting.
I20260811 14:35:23.927868 4249 tablet_bootstrap.cc:492] T e6f97aee9ce341aeb0cb64665441253f P f6a8b39849444152a003f5b71e7aee42: No bootstrap required, opened a new log
I20260811 14:35:23.927927 4249 ts_tablet_manager.cc:1397] T e6f97aee9ce341aeb0cb64665441253f P f6a8b39849444152a003f5b71e7aee42: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 14:35:23.928345 4247 tablet_bootstrap.cc:654] T e6f97aee9ce341aeb0cb64665441253f P 1c0ec704dd254432a62e274805280548: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:23.928876 4247 log.cc:824] T e6f97aee9ce341aeb0cb64665441253f P 1c0ec704dd254432a62e274805280548: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:23.929389 4249 raft_consensus.cc:348] T e6f97aee9ce341aeb0cb64665441253f P f6a8b39849444152a003f5b71e7aee42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0ec704dd254432a62e274805280548" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35503 } } peers { permanent_uuid: "f6a8b39849444152a003f5b71e7aee42" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 38515 } } peers { permanent_uuid: "12aef800531d4b458021b3fea32d664f" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 43607 } }
I20260811 14:35:23.929538 4249 raft_consensus.cc:374] T e6f97aee9ce341aeb0cb64665441253f P f6a8b39849444152a003f5b71e7aee42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:23.929561 4249 raft_consensus.cc:729] T e6f97aee9ce341aeb0cb64665441253f P f6a8b39849444152a003f5b71e7aee42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6a8b39849444152a003f5b71e7aee42, State: Initialized, Role: FOLLOWER
I20260811 14:35:23.929665 4249 consensus_queue.cc:260] T e6f97aee9ce341aeb0cb64665441253f P f6a8b39849444152a003f5b71e7aee42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0ec704dd254432a62e274805280548" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35503 } } peers { permanent_uuid: "f6a8b39849444152a003f5b71e7aee42" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 38515 } } peers { permanent_uuid: "12aef800531d4b458021b3fea32d664f" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 43607 } }
I20260811 14:35:23.929718 4247 tablet_bootstrap.cc:492] T e6f97aee9ce341aeb0cb64665441253f P 1c0ec704dd254432a62e274805280548: No bootstrap required, opened a new log
I20260811 14:35:23.929787 4247 ts_tablet_manager.cc:1397] T e6f97aee9ce341aeb0cb64665441253f P 1c0ec704dd254432a62e274805280548: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:35:23.929965 4249 ts_tablet_manager.cc:1428] T e6f97aee9ce341aeb0cb64665441253f P f6a8b39849444152a003f5b71e7aee42: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 14:35:23.930155 4001 heartbeater.cc:503] Master 127.2.103.126:41121 was elected leader, sending a full tablet report...
I20260811 14:35:23.930245 4245 log.cc:824] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:23.931157 4245 tablet_bootstrap.cc:492] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f: No bootstrap required, opened a new log
I20260811 14:35:23.931231 4245 ts_tablet_manager.cc:1397] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 14:35:23.931998 4247 raft_consensus.cc:348] T e6f97aee9ce341aeb0cb64665441253f P 1c0ec704dd254432a62e274805280548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0ec704dd254432a62e274805280548" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35503 } } peers { permanent_uuid: "f6a8b39849444152a003f5b71e7aee42" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 38515 } } peers { permanent_uuid: "12aef800531d4b458021b3fea32d664f" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 43607 } }
I20260811 14:35:23.932165 4247 raft_consensus.cc:374] T e6f97aee9ce341aeb0cb64665441253f P 1c0ec704dd254432a62e274805280548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:23.932212 4247 raft_consensus.cc:729] T e6f97aee9ce341aeb0cb64665441253f P 1c0ec704dd254432a62e274805280548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c0ec704dd254432a62e274805280548, State: Initialized, Role: FOLLOWER
I20260811 14:35:23.932331 4247 consensus_queue.cc:260] T e6f97aee9ce341aeb0cb64665441253f P 1c0ec704dd254432a62e274805280548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0ec704dd254432a62e274805280548" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35503 } } peers { permanent_uuid: "f6a8b39849444152a003f5b71e7aee42" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 38515 } } peers { permanent_uuid: "12aef800531d4b458021b3fea32d664f" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 43607 } }
I20260811 14:35:23.932616 4247 ts_tablet_manager.cc:1428] T e6f97aee9ce341aeb0cb64665441253f P 1c0ec704dd254432a62e274805280548: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:35:23.932662 4212 heartbeater.cc:503] Master 127.2.103.126:41121 was elected leader, sending a full tablet report...
I20260811 14:35:23.933354 4245 raft_consensus.cc:348] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0ec704dd254432a62e274805280548" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35503 } } peers { permanent_uuid: "f6a8b39849444152a003f5b71e7aee42" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 38515 } } peers { permanent_uuid: "12aef800531d4b458021b3fea32d664f" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 43607 } }
I20260811 14:35:23.933547 4245 raft_consensus.cc:374] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:23.933595 4245 raft_consensus.cc:729] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12aef800531d4b458021b3fea32d664f, State: Initialized, Role: FOLLOWER
I20260811 14:35:23.933705 4245 consensus_queue.cc:260] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0ec704dd254432a62e274805280548" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35503 } } peers { permanent_uuid: "f6a8b39849444152a003f5b71e7aee42" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 38515 } } peers { permanent_uuid: "12aef800531d4b458021b3fea32d664f" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 43607 } }
I20260811 14:35:23.934064 4245 ts_tablet_manager.cc:1428] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:35:23.934122 3767 heartbeater.cc:503] Master 127.2.103.126:41121 was elected leader, sending a full tablet report...
W20260811 14:35:23.962836 4002 tablet.cc:2369] T e6f97aee9ce341aeb0cb64665441253f P f6a8b39849444152a003f5b71e7aee42: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:35:24.027619 4255 raft_consensus.cc:482] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:35:24.027766 4255 raft_consensus.cc:504] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0ec704dd254432a62e274805280548" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35503 } } peers { permanent_uuid: "f6a8b39849444152a003f5b71e7aee42" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 38515 } } peers { permanent_uuid: "12aef800531d4b458021b3fea32d664f" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 43607 } }
I20260811 14:35:24.028519 4255 leader_election.cc:288] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c0ec704dd254432a62e274805280548 (127.2.103.67:35503), f6a8b39849444152a003f5b71e7aee42 (127.2.103.66:38515)
W20260811 14:35:24.028674 3770 tablet.cc:2369] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:35:24.031896 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:24.028692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:35503 (local address 127.2.103.65:41499)
0811 14:35:24.028939 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:24.028951 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:24.029200 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:24.029370 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:24.029370 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:24.029372 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:24.029474 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:24.029476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:24.030140 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:24.030141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:24.031019 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:24.031022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:24.031607 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:24.031608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:24.031688 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:24.031697 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 14:35:24.031741 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 14:35:24.031777 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":10}
I20260811 14:35:24.031896 4242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:24.028987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:41499 (local address 127.2.103.67:35503)
0811 14:35:24.029036 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:24.029038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:24.029250 (+ 212us) server_negotiation.cc:407] Connection header received
0811 14:35:24.029281 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:24.029282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:24.029292 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:24.029347 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:24.029558 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:24.030046 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:24.031114 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:24.031517 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:24.031577 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:24.031586 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:24.031771 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 14:35:24.031825 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 14:35:24.031997 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:24.028825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:54645 (local address 127.2.103.66:38515)
0811 14:35:24.028904 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:24.028907 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:24.029041 (+ 134us) server_negotiation.cc:407] Connection header received
0811 14:35:24.029117 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:24.029118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:24.029137 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:24.029208 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:24.029435 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:24.029982 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:24.031153 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:24.031538 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:24.031638 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:24.031648 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:24.031873 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 14:35:24.031921 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 14:35:24.032017 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:24.028786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:38515 (local address 127.2.103.65:54645)
0811 14:35:24.028944 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:24.028957 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:24.029024 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:24.029247 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:24.029249 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:24.029251 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:24.029343 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:24.029345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:24.030082 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:24.030083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:24.031017 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:24.031019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:24.031657 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:24.031658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:24.031769 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:24.031773 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:35:24.031832 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 14:35:24.031872 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":49,"thread_start_us":87,"threads_started":1}
I20260811 14:35:24.032505 4143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6f97aee9ce341aeb0cb64665441253f" candidate_uuid: "12aef800531d4b458021b3fea32d664f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c0ec704dd254432a62e274805280548" is_pre_election: true
I20260811 14:35:24.032683 4143 raft_consensus.cc:2454] T e6f97aee9ce341aeb0cb64665441253f P 1c0ec704dd254432a62e274805280548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12aef800531d4b458021b3fea32d664f in term 0.
I20260811 14:35:24.033018 3587 leader_election.cc:302] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12aef800531d4b458021b3fea32d664f, 1c0ec704dd254432a62e274805280548; no voters:
I20260811 14:35:24.033246 4255 raft_consensus.cc:2781] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:35:24.033324 4255 raft_consensus.cc:482] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:35:24.033344 4255 raft_consensus.cc:3037] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:24.034086 4255 raft_consensus.cc:504] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0ec704dd254432a62e274805280548" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35503 } } peers { permanent_uuid: "f6a8b39849444152a003f5b71e7aee42" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 38515 } } peers { permanent_uuid: "12aef800531d4b458021b3fea32d664f" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 43607 } }
I20260811 14:35:24.034294 4255 leader_election.cc:288] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f [CANDIDATE]: Term 1 election: Requested vote from peers 1c0ec704dd254432a62e274805280548 (127.2.103.67:35503), f6a8b39849444152a003f5b71e7aee42 (127.2.103.66:38515)
I20260811 14:35:24.034382 4143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6f97aee9ce341aeb0cb64665441253f" candidate_uuid: "12aef800531d4b458021b3fea32d664f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c0ec704dd254432a62e274805280548"
I20260811 14:35:24.034449 4143 raft_consensus.cc:3037] T e6f97aee9ce341aeb0cb64665441253f P 1c0ec704dd254432a62e274805280548 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:24.034507 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6f97aee9ce341aeb0cb64665441253f" candidate_uuid: "12aef800531d4b458021b3fea32d664f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6a8b39849444152a003f5b71e7aee42"
I20260811 14:35:24.034672 3922 raft_consensus.cc:3037] T e6f97aee9ce341aeb0cb64665441253f P f6a8b39849444152a003f5b71e7aee42 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:24.034852 3923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6f97aee9ce341aeb0cb64665441253f" candidate_uuid: "12aef800531d4b458021b3fea32d664f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6a8b39849444152a003f5b71e7aee42" is_pre_election: true
I20260811 14:35:24.035462 4143 raft_consensus.cc:2454] T e6f97aee9ce341aeb0cb64665441253f P 1c0ec704dd254432a62e274805280548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12aef800531d4b458021b3fea32d664f in term 1.
I20260811 14:35:24.035624 3587 leader_election.cc:302] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12aef800531d4b458021b3fea32d664f, 1c0ec704dd254432a62e274805280548; no voters:
I20260811 14:35:24.035637 3922 raft_consensus.cc:2454] T e6f97aee9ce341aeb0cb64665441253f P f6a8b39849444152a003f5b71e7aee42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12aef800531d4b458021b3fea32d664f in term 1.
I20260811 14:35:24.035735 4255 raft_consensus.cc:2781] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:24.035930 4255 raft_consensus.cc:686] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f [term 1 LEADER]: Becoming Leader. State: Replica: 12aef800531d4b458021b3fea32d664f, State: Running, Role: LEADER
I20260811 14:35:24.036062 4255 consensus_queue.cc:237] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0ec704dd254432a62e274805280548" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35503 } } peers { permanent_uuid: "f6a8b39849444152a003f5b71e7aee42" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 38515 } } peers { permanent_uuid: "12aef800531d4b458021b3fea32d664f" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 43607 } }
I20260811 14:35:24.037248 3451 catalog_manager.cc:5495] T e6f97aee9ce341aeb0cb64665441253f P 12aef800531d4b458021b3fea32d664f reported cstate change: term changed from 0 to 1, leader changed from <none> to 12aef800531d4b458021b3fea32d664f (127.2.103.65). New cstate: current_term: 1 leader_uuid: "12aef800531d4b458021b3fea32d664f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0ec704dd254432a62e274805280548" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6a8b39849444152a003f5b71e7aee42" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 38515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12aef800531d4b458021b3fea32d664f" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 43607 } health_report { overall_health: HEALTHY } } }
Aug 11 14:35:24 dist-test-slave-twwt krb5kdc[3380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.65@KRBTEST.COM
I20260811 14:35:24.073166 4244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:24.065475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49938 (local address 127.2.103.65:43607)
0811 14:35:24.065581 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:24.065583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:24.065683 (+ 100us) server_negotiation.cc:407] Connection header received
0811 14:35:24.065742 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:24.065742 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:24.065753 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:24.065805 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:24.066376 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:24.066930 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:24.067649 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:24.067834 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:24.067895 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:24.070268 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:24.070282 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:24.070282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:24.070289 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:24.072083 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:24.072418 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:24.072421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:24.072422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:24.072478 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:24.072648 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:24.072649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:24.072649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:24.072796 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:24.072907 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:24.072971 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 14:35:24.073037 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
Aug 11 14:35:24 dist-test-slave-twwt krb5kdc[3380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458924, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 14:35:24 dist-test-slave-twwt krb5kdc[3380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458924, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.103.65@KRBTEST.COM
I20260811 14:35:24.101544 4244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:24.094267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49948 (local address 127.2.103.65:43607)
0811 14:35:24.094360 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:24.094362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:24.094405 (+ 43us) server_negotiation.cc:407] Connection header received
0811 14:35:24.094465 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:24.094466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:24.094477 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:24.094531 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:24.095165 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:24.095654 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:24.096378 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:24.096574 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:24.096631 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:24.098971 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:24.098973 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:24.098973 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:24.098980 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:24.100392 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:24.100751 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:24.100753 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:24.100753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:24.100817 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:24.101013 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:24.101013 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:24.101014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:24.101158 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:24.101251 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:24.101403 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 14:35:24.101474 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 14:35:24 dist-test-slave-twwt krb5kdc[3380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 14:35:24 dist-test-slave-twwt krb5kdc[3380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.65@KRBTEST.COM
I20260811 14:35:24.131464 4244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:24.123391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49960 (local address 127.2.103.65:43607)
0811 14:35:24.123448 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:24.123450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:24.123568 (+ 118us) server_negotiation.cc:407] Connection header received
0811 14:35:24.123664 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:24.123665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:24.123676 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:24.123730 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:24.124393 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:24.124893 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:24.125679 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:24.125896 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:24.125984 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:24.128508 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:24.128510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:24.128510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:24.128518 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:24.130098 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:24.130461 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:24.130462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:24.130462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:24.130517 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:24.130881 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:24.130882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:24.130882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:24.131002 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:24.131121 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:24.131275 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 14:35:24.131345 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260811 14:35:24.132331 3628 tablet_service.cc:2908] Rejecting scan request for tablet e6f97aee9ce341aeb0cb64665441253f: Uninitialized: clean time has not yet been initialized
I20260811 14:35:24.133502 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 3560
W20260811 14:35:24.136727 4215 tablet.cc:2369] T e6f97aee9ce341aeb0cb64665441253f P 1c0ec704dd254432a62e274805280548: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:35:24.141871 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 3776
I20260811 14:35:24.154241 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 4006
I20260811 14:35:24.170681 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 3402
2026-08-11T14:35:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2484 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:24 dist-test-slave-twwt krb5kdc[4284](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:24 dist-test-slave-twwt krb5kdc[4284](info): set up 2 sockets
Aug 11 14:35:24 dist-test-slave-twwt krb5kdc[4284](info): commencing operation
krb5kdc: starting...
W20260811 14:35:25.493788 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:35:25 dist-test-slave-twwt krb5kdc[4284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:35:25.638833 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:33913
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:41317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:33913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:25.731688 4299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:25.732039 4299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:25.732093 4299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:25.733731 4299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:25.733855 4299 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 14:35:25.733883 4299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:25.733908 4299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:25.733939 4299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:25.733961 4299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:25.735863 4299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:41317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:33913
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:33913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:25.736240 4299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:25.736526 4299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:25.742676 4309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:25.742656 4308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:25.742690 4299 server_base.cc:1034] running on GCE node
W20260811 14:35:25.742666 4312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:25.743319 4299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:25.743707 4299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:25.744938 4299 hybrid_clock.cc:648] HybridClock initialized: now 1786458925744896 us; error 61 us; skew 500 ppm
Aug 11 14:35:25 dist-test-slave-twwt krb5kdc[4284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458925, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:25.748327 4299 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:35:25.749040 4299 webserver.cc:460] Webserver started at http://127.2.103.126:35175/ using document root <none> and password file <none>
I20260811 14:35:25.749450 4299 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:25.749568 4299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:25.749764 4299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:25.751439 4299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "dfbd7aa767f54a0b82585f955efafe68"
format_stamp: "Formatted at 2026-08-11 14:35:25 on dist-test-slave-twwt"
server_key: "b5ecf58d114813131016367bf8529084"
server_key_iv: "d5d702c062530c6a5318e30dd2a7c167"
server_key_version: "encryptionkey@0"
I20260811 14:35:25.751909 4299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "dfbd7aa767f54a0b82585f955efafe68"
format_stamp: "Formatted at 2026-08-11 14:35:25 on dist-test-slave-twwt"
server_key: "b5ecf58d114813131016367bf8529084"
server_key_iv: "d5d702c062530c6a5318e30dd2a7c167"
server_key_version: "encryptionkey@0"
I20260811 14:35:25.753903 4299 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 14:35:25.755141 4321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:25.755417 4299 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:25.755559 4299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "dfbd7aa767f54a0b82585f955efafe68"
format_stamp: "Formatted at 2026-08-11 14:35:25 on dist-test-slave-twwt"
server_key: "b5ecf58d114813131016367bf8529084"
server_key_iv: "d5d702c062530c6a5318e30dd2a7c167"
server_key_version: "encryptionkey@0"
I20260811 14:35:25.755659 4299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:25.769922 4299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:25.772794 4299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:25.773110 4299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:25.780047 4299 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:33913
I20260811 14:35:25.780105 4426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:33913 every 8 connection(s)
I20260811 14:35:25.780550 4299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:25.781675 4427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:25.784466 4427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68: Bootstrap starting.
I20260811 14:35:25.785099 4427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:25.785364 4427 log.cc:824] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:25.785878 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 4299
I20260811 14:35:25.786018 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:35:25.786221 2461 external_mini_cluster.cc:1428] Setting key 9fc6dfa73b6239393a3c1c51d278baae
I20260811 14:35:25.786247 4427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68: No bootstrap required, opened a new log
I20260811 14:35:25.788993 4427 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbd7aa767f54a0b82585f955efafe68" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33913 } }
I20260811 14:35:25.789155 4427 raft_consensus.cc:374] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:25.789179 4427 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfbd7aa767f54a0b82585f955efafe68, State: Initialized, Role: FOLLOWER
I20260811 14:35:25.789333 4427 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbd7aa767f54a0b82585f955efafe68" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33913 } }
I20260811 14:35:25.789420 4427 raft_consensus.cc:388] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:25.789467 4427 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:25.789518 4427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:25.790159 4427 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbd7aa767f54a0b82585f955efafe68" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33913 } }
I20260811 14:35:25.790304 4427 leader_election.cc:302] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfbd7aa767f54a0b82585f955efafe68; no voters:
I20260811 14:35:25.790518 4427 leader_election.cc:288] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:25.790596 4435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:25.790831 4435 raft_consensus.cc:686] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68 [term 1 LEADER]: Becoming Leader. State: Replica: dfbd7aa767f54a0b82585f955efafe68, State: Running, Role: LEADER
I20260811 14:35:25.790899 4427 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:25.790974 4435 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbd7aa767f54a0b82585f955efafe68" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33913 } }
I20260811 14:35:25.791364 4435 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfbd7aa767f54a0b82585f955efafe68. Latest consensus state: current_term: 1 leader_uuid: "dfbd7aa767f54a0b82585f955efafe68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbd7aa767f54a0b82585f955efafe68" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33913 } } }
I20260811 14:35:25.791446 4435 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:25.791378 4437 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dfbd7aa767f54a0b82585f955efafe68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbd7aa767f54a0b82585f955efafe68" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33913 } } }
I20260811 14:35:25.791657 4437 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:25.792114 4444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:25.792752 4444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:25.795686 4444 catalog_manager.cc:1347] Generated new cluster ID: c571a5942e8d4dd2b115cc6953625455
I20260811 14:35:25.795791 4444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:35:25 dist-test-slave-twwt krb5kdc[4284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:25.801302 4444 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:25.801671 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:25.786967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48780 (local address 127.2.103.126:33913)
0811 14:35:25.787193 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:25.787203 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:25.787328 (+ 125us) server_negotiation.cc:407] Connection header received
0811 14:35:25.788981 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:25.788992 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:25.789058 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:25.789260 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:25.790147 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:25.791882 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:25.793021 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:25.793640 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:25.793806 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:25.797385 (+ 3579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:25.797390 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:25.797391 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:25.797406 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:25.799630 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:25.800117 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:25.800121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:25.800122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:25.800318 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:25.800583 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:25.800583 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:25.800584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:25.800931 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:25.801103 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:25.801277 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 14:35:25.801444 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":129,"threads_started":1}
I20260811 14:35:25.802050 4444 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:25.806583 4444 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68: Generated new TSK 0
I20260811 14:35:25.806864 4444 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
I20260811 14:35:25.847000 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:33913
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:41317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:25.935627 4473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:25.935856 4473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:25.935883 4473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:25.937093 4473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:25.937131 4473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:25.937152 4473 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 14:35:25.937171 4473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:35:25.938704 4473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:41317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.103.126:33913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:25.938930 4473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:25.939113 4473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:25.944600 4480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:25.944804 4485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:25.944804 4482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:25.945788 4473 server_base.cc:1034] running on GCE node
I20260811 14:35:25.946056 4473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:25.946355 4473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:25.947546 4473 hybrid_clock.cc:648] HybridClock initialized: now 1786458925947517 us; error 52 us; skew 500 ppm
Aug 11 14:35:25 dist-test-slave-twwt krb5kdc[4284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458925, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:25.951138 4473 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:35:25.951761 4473 webserver.cc:460] Webserver started at http://127.2.103.65:46193/ using document root <none> and password file <none>
I20260811 14:35:25.952095 4473 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:25.952153 4473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:25.952286 4473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:25.953781 4473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "cb2428b7f8664aa6ab5db6e748e0d607"
format_stamp: "Formatted at 2026-08-11 14:35:25 on dist-test-slave-twwt"
server_key: "917fbd1c5d3aff40418107c95bbd2863"
server_key_iv: "d4e0808d1a5c802afca1021d14e3017d"
server_key_version: "encryptionkey@0"
I20260811 14:35:25.954196 4473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "cb2428b7f8664aa6ab5db6e748e0d607"
format_stamp: "Formatted at 2026-08-11 14:35:25 on dist-test-slave-twwt"
server_key: "917fbd1c5d3aff40418107c95bbd2863"
server_key_iv: "d4e0808d1a5c802afca1021d14e3017d"
server_key_version: "encryptionkey@0"
I20260811 14:35:25.958789 4473 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 14:35:25.959940 4496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:25.960108 4473 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:35:25.960201 4473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "cb2428b7f8664aa6ab5db6e748e0d607"
format_stamp: "Formatted at 2026-08-11 14:35:25 on dist-test-slave-twwt"
server_key: "917fbd1c5d3aff40418107c95bbd2863"
server_key_iv: "d4e0808d1a5c802afca1021d14e3017d"
server_key_version: "encryptionkey@0"
I20260811 14:35:25.960269 4473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:25.971351 4473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:25.974023 4473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:25.974229 4473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:25.974927 4473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:25.975006 4473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:25.975052 4473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:25.975083 4473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:25.983868 4473 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:35989
I20260811 14:35:25.983958 4690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:35989 every 8 connection(s)
Aug 11 14:35:25 dist-test-slave-twwt krb5kdc[4284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458925, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:25.984357 4473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:35:25.989019 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:25.975780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:59641 (local address 127.2.103.126:33913)
0811 14:35:25.975853 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:25.975856 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:25.976082 (+ 226us) server_negotiation.cc:407] Connection header received
0811 14:35:25.976770 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:25.976772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:25.976793 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:25.976873 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:25.977896 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:25.979002 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:25.981128 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:25.981531 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:25.981646 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:25.985069 (+ 3423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:25.985071 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:25.985071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:25.985086 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:25.986924 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:25.987481 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:25.987483 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:25.987483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:25.987549 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:25.987930 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:25.987932 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:25.987932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:25.988080 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:25.988152 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:25.988719 (+ 567us) server_negotiation.cc:299] Negotiation successful
0811 14:35:25.988907 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 14:35:25.989023 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 4473
I20260811 14:35:25.989125 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:35:25.989087 4510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:25.975705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:33913 (local address 127.2.103.65:59641)
0811 14:35:25.975943 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:25.975983 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:25.976593 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:25.976988 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:25.976992 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:25.977410 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:25.977743 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:25.977752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:25.979172 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:25.979174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:25.980965 (+ 1791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:25.980972 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:25.981699 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:25.981700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:25.981972 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:25.982873 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:25.982894 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:25.984942 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:25.987042 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:25.987050 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:25.987053 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:25.987320 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:25.987655 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:25.987656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:25.987656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:25.987791 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:25.988184 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:25.988192 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:25.988359 (+ 167us) client_negotiation.cc:769] Sending connection context
0811 14:35:25.988484 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 14:35:25.988888 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":94,"threads_started":1}
I20260811 14:35:25.989346 2461 external_mini_cluster.cc:1428] Setting key bb5597367710d56a6bab2de371970249
I20260811 14:35:25.990156 4693 heartbeater.cc:347] Connected to a master server at 127.2.103.126:33913
I20260811 14:35:25.990310 4693 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:25.990569 4693 heartbeater.cc:511] Master 127.2.103.126:33913 requested a full tablet report, sending...
I20260811 14:35:25.991297 4344 ts_manager.cc:194] Registered new tserver with Master: cb2428b7f8664aa6ab5db6e748e0d607 (127.2.103.65:35989)
I20260811 14:35:25.992419 4344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:59641
I20260811 14:35:25.999286 4700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:25.991465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:33913 (local address 127.2.103.65:41139)
0811 14:35:25.991646 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:25.991659 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:25.991740 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:25.991969 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:25.991970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:25.992225 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:25.992311 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:25.992313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:25.993368 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:25.993369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:25.994211 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:25.994214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:25.994801 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:25.994802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:25.994878 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:25.995506 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:25.995516 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:25.995903 (+ 387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:25.998090 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:25.998091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:25.998092 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:25.998290 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:25.998544 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:25.998545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:25.998545 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:25.998666 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:25.998985 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:25.998986 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:25.999036 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 14:35:25.999097 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 14:35:25.999198 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":96,"threads_started":1}
I20260811 14:35:25.999400 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:25.991513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:41139 (local address 127.2.103.126:33913)
0811 14:35:25.991554 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:25.991556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:25.991714 (+ 158us) server_negotiation.cc:407] Connection header received
0811 14:35:25.991855 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:25.991856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:25.991869 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:25.991960 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:25.992431 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:25.993236 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:25.994322 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:25.994705 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:25.994787 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:25.996000 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:25.996002 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:25.996002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:25.996010 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:25.997985 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:25.998389 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:25.998391 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:25.998392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:25.998464 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:25.998739 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:25.998740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:25.998740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:25.998903 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:25.998982 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:25.999156 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 14:35:25.999267 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.2.103.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
I20260811 14:35:26.026226 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:33913
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:41317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:26.115037 4704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:26.115254 4704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:26.115280 4704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:26.116448 4704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:26.116489 4704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:26.116510 4704 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 14:35:26.116528 4704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:35:26.117985 4704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:41317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.103.126:33913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:26.118197 4704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:26.118392 4704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:26.124063 4710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:26.124118 4704 server_base.cc:1034] running on GCE node
W20260811 14:35:26.124063 4713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:26.124082 4711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:26.124498 4704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:26.124775 4704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:26.125985 4704 hybrid_clock.cc:648] HybridClock initialized: now 1786458926125977 us; error 40 us; skew 500 ppm
Aug 11 14:35:26 dist-test-slave-twwt krb5kdc[4284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458926, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:26.129009 4704 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:35:26.129550 4704 webserver.cc:460] Webserver started at http://127.2.103.66:45781/ using document root <none> and password file <none>
I20260811 14:35:26.129841 4704 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:26.129925 4704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:26.130102 4704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:26.131393 4704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "0ca1dae499ef45ab957f06a7c2808f52"
format_stamp: "Formatted at 2026-08-11 14:35:26 on dist-test-slave-twwt"
server_key: "f391b5295cd1fd6981aea2671191d238"
server_key_iv: "ad3a021be992779b80f5e7f3dcc0c551"
server_key_version: "encryptionkey@0"
I20260811 14:35:26.131812 4704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "0ca1dae499ef45ab957f06a7c2808f52"
format_stamp: "Formatted at 2026-08-11 14:35:26 on dist-test-slave-twwt"
server_key: "f391b5295cd1fd6981aea2671191d238"
server_key_iv: "ad3a021be992779b80f5e7f3dcc0c551"
server_key_version: "encryptionkey@0"
I20260811 14:35:26.133391 4704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:35:26.134265 4721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:26.134455 4704 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:26.134577 4704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "0ca1dae499ef45ab957f06a7c2808f52"
format_stamp: "Formatted at 2026-08-11 14:35:26 on dist-test-slave-twwt"
server_key: "f391b5295cd1fd6981aea2671191d238"
server_key_iv: "ad3a021be992779b80f5e7f3dcc0c551"
server_key_version: "encryptionkey@0"
I20260811 14:35:26.134729 4704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:26.159024 4704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:26.162464 4704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:26.162711 4704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:26.163380 4704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:26.163460 4704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:26.163520 4704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:26.163555 4704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 14:35:26 dist-test-slave-twwt krb5kdc[4284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458926, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:26.172189 4704 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:45653
I20260811 14:35:26.172290 4915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:45653 every 8 connection(s)
I20260811 14:35:26.172792 4704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:35:26.173085 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 4704
I20260811 14:35:26.173209 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:35:26.173451 2461 external_mini_cluster.cc:1428] Setting key d9bb9f0376fbd743ab84884d3bbbf812
I20260811 14:35:26.176119 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.164113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:55757 (local address 127.2.103.126:33913)
0811 14:35:26.164193 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:26.164196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:26.164473 (+ 277us) server_negotiation.cc:407] Connection header received
0811 14:35:26.165189 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:26.165191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:26.165205 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:26.165276 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:26.166356 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.167374 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.168739 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.169014 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.169111 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.172052 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:26.172054 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:26.172055 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:26.172064 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:26.174290 (+ 2226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:26.174926 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:26.174928 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:26.174928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:26.175009 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:26.175334 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:26.175335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:26.175336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:26.175482 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:26.175570 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:26.175901 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 14:35:26.176014 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 14:35:26.176554 4735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.164105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:33913 (local address 127.2.103.66:55757)
0811 14:35:26.164325 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:26.164368 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:26.165054 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:26.165384 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:26.165389 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:26.165856 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:26.166208 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:26.166217 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.167515 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.167516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:26.168601 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:26.168604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.169138 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.169139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:26.169306 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.169967 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:26.169977 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:26.171936 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:26.174395 (+ 2459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:26.174403 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:26.174406 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:26.174828 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:26.175091 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:26.175092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:26.175092 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:26.175244 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:26.175561 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:26.175569 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:26.175738 (+ 169us) client_negotiation.cc:769] Sending connection context
0811 14:35:26.175873 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 14:35:26.176076 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
I20260811 14:35:26.177150 4917 heartbeater.cc:347] Connected to a master server at 127.2.103.126:33913
I20260811 14:35:26.177254 4917 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:26.177496 4917 heartbeater.cc:511] Master 127.2.103.126:33913 requested a full tablet report, sending...
I20260811 14:35:26.177975 4345 ts_manager.cc:194] Registered new tserver with Master: 0ca1dae499ef45ab957f06a7c2808f52 (127.2.103.66:45653)
I20260811 14:35:26.178436 4345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:55757
WARNING: no policy specified for kudu/127.2.103.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.67@KRBTEST.COM" created.
I20260811 14:35:26.190116 4927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.179216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:33913 (local address 127.2.103.66:58685)
0811 14:35:26.182789 (+ 3573us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:26.182803 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:26.182898 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:26.183083 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:26.183084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:26.183279 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:26.183345 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:26.183346 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.184385 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.184387 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:26.184950 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:26.184953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.185498 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.185499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:26.185581 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.186161 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:26.186167 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:26.186584 (+ 417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:26.188631 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:26.188633 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:26.188633 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:26.188905 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:26.189147 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:26.189149 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:26.189149 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:26.189228 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:26.189709 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:26.189710 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:26.189764 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 14:35:26.189867 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 14:35:26.190002 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3533,"thread_start_us":77,"threads_started":1}
I20260811 14:35:26.190104 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.179124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:58685 (local address 127.2.103.126:33913)
0811 14:35:26.179218 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:26.179220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:26.182911 (+ 3691us) server_negotiation.cc:407] Connection header received
0811 14:35:26.182983 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:26.182984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:26.182999 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:26.183069 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:26.183443 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.184260 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.185061 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.185378 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.185465 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.186830 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:26.186832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:26.186832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:26.186840 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:26.188525 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:26.188999 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:26.189000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:26.189001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:26.189066 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:26.189326 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:26.189327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:26.189328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:26.189562 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:26.189677 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:26.189900 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 14:35:26.189999 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
I20260811 14:35:26.211227 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:33913
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:41317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:26.308552 4933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:26.308789 4933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:26.308816 4933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:26.310141 4933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:26.310204 4933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:26.310227 4933 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 14:35:26.310246 4933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:35:26.312423 4933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:41317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.103.126:33913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:26.312741 4933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:26.313045 4933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:26.318770 4941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:26.318900 4933 server_base.cc:1034] running on GCE node
W20260811 14:35:26.318845 4942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:26.319083 4944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:26.319386 4933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:26.319655 4933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:26.320854 4933 hybrid_clock.cc:648] HybridClock initialized: now 1786458926320789 us; error 64 us; skew 500 ppm
Aug 11 14:35:26 dist-test-slave-twwt krb5kdc[4284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458926, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:26.324002 4933 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:35:26.324636 4933 webserver.cc:460] Webserver started at http://127.2.103.67:36457/ using document root <none> and password file <none>
I20260811 14:35:26.324921 4933 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:26.325003 4933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:26.325173 4933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:26.326419 4933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "d8078449ee3942d8b30c8d2b0152c43f"
format_stamp: "Formatted at 2026-08-11 14:35:26 on dist-test-slave-twwt"
server_key: "fd970a733a035a38708cf6ceb1941016"
server_key_iv: "b550c5fc0e594b27a09d2bebb42dc29e"
server_key_version: "encryptionkey@0"
I20260811 14:35:26.326916 4933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "d8078449ee3942d8b30c8d2b0152c43f"
format_stamp: "Formatted at 2026-08-11 14:35:26 on dist-test-slave-twwt"
server_key: "fd970a733a035a38708cf6ceb1941016"
server_key_iv: "b550c5fc0e594b27a09d2bebb42dc29e"
server_key_version: "encryptionkey@0"
I20260811 14:35:26.328635 4933 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:35:26.329727 4958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:26.330013 4933 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260811 14:35:26.330120 4933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "d8078449ee3942d8b30c8d2b0152c43f"
format_stamp: "Formatted at 2026-08-11 14:35:26 on dist-test-slave-twwt"
server_key: "fd970a733a035a38708cf6ceb1941016"
server_key_iv: "b550c5fc0e594b27a09d2bebb42dc29e"
server_key_version: "encryptionkey@0"
I20260811 14:35:26.330235 4933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:26.347704 4933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:26.350425 4933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:26.350557 4933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:26.351254 4933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:26.351362 4933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:26.351430 4933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:26.351462 4933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 14:35:26 dist-test-slave-twwt krb5kdc[4284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458926, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:26.359450 4933 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:43551
I20260811 14:35:26.359506 5141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:43551 every 8 connection(s)
I20260811 14:35:26.359889 4933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:35:26.363340 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.351921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:40861 (local address 127.2.103.126:33913)
0811 14:35:26.352002 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:26.352005 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:26.352255 (+ 250us) server_negotiation.cc:407] Connection header received
0811 14:35:26.352853 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:26.352855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:26.352872 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:26.352959 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:26.353904 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.354866 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.356498 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.356839 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.356949 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.360004 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:26.360006 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:26.360006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:26.360016 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:26.361683 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:26.362175 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:26.362176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:26.362176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:26.362239 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:26.362525 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:26.362526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:26.362527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:26.362757 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:26.362833 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:26.363158 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 14:35:26.363253 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 14:35:26.363411 4973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.351902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:33913 (local address 127.2.103.67:40861)
0811 14:35:26.352109 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:26.352154 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:26.352686 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:26.353038 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:26.353042 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:26.353460 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:26.353774 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:26.353780 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.354993 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.354994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:26.356406 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:26.356417 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.356962 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.356963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:26.357115 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.357801 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:26.357819 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:26.359868 (+ 2049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:26.361819 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:26.361827 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:26.361829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:26.362085 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:26.362315 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:26.362316 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:26.362316 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:26.362430 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:26.362825 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:26.362831 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:26.362971 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 14:35:26.363073 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 14:35:26.363221 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":130,"threads_started":1}
I20260811 14:35:26.363987 5142 heartbeater.cc:347] Connected to a master server at 127.2.103.126:33913
I20260811 14:35:26.364126 5142 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:26.364466 5142 heartbeater.cc:511] Master 127.2.103.126:33913 requested a full tablet report, sending...
I20260811 14:35:26.364934 4344 ts_manager.cc:194] Registered new tserver with Master: d8078449ee3942d8b30c8d2b0152c43f (127.2.103.67:43551)
I20260811 14:35:26.365415 4344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:40861
I20260811 14:35:26.368798 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 4933
I20260811 14:35:26.368916 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:35:26.369104 2461 external_mini_cluster.cc:1428] Setting key d7bd2059102970125aa6dce49bbe3a3c
I20260811 14:35:26.370085 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:35:26.372421 5152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.365191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:33913 (local address 127.2.103.67:46729)
0811 14:35:26.365460 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:26.365476 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:26.365591 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:26.365777 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:26.365778 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:26.366022 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:26.366092 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:26.366095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.366915 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.366917 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:26.367503 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:26.367505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.367916 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.367917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:26.367984 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.368603 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:26.368609 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:26.369031 (+ 422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:26.371207 (+ 2176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:26.371208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:26.371208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:26.371422 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:26.371676 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:26.371677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:26.371677 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:26.371750 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:26.372091 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:26.372092 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:26.372140 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 14:35:26.372201 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 14:35:26.372336 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":135,"threads_started":1}
I20260811 14:35:26.372895 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.365377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:46729 (local address 127.2.103.126:33913)
0811 14:35:26.365440 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:26.365442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:26.365570 (+ 128us) server_negotiation.cc:407] Connection header received
0811 14:35:26.365657 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:26.365658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:26.365673 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:26.365759 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:26.366177 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.366716 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.367597 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.367836 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.367896 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.369135 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:26.369137 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:26.369137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:26.369146 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:26.371095 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:26.371516 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:26.371518 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:26.371518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:26.371594 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:26.371838 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:26.371839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:26.371839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:26.372010 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:26.372089 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:26.372262 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 14:35:26.372366 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Aug 11 14:35:26 dist-test-slave-twwt krb5kdc[4284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458926, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 14:35:26 dist-test-slave-twwt krb5kdc[4284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458926, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:26.399595 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.390918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48794 (local address 127.2.103.126:33913)
0811 14:35:26.391022 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:26.391024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:26.391079 (+ 55us) server_negotiation.cc:407] Connection header received
0811 14:35:26.391191 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:26.391193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:26.391207 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:26.391276 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:26.392132 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.392781 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.393616 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.393858 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.393925 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.396569 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:26.396571 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:26.396572 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:26.396581 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:26.398103 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:26.398577 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:26.398579 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:26.398579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:26.398698 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:26.398950 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:26.398951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:26.398951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:26.399091 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:26.399216 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:26.399369 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 14:35:26.399475 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 14:35:26.401764 4344 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:35:26.403640 4344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:35:26.411726 5170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.406683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46258 (local address 127.2.103.65:35989)
0811 14:35:26.407424 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:26.407429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:26.407440 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:35:26.407483 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:26.407485 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:26.407500 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:26.407594 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:26.407771 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.408676 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.410468 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.411161 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.411279 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.411352 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:26.411568 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 14:35:26.411610 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":121,"threads_started":1}
I20260811 14:35:26.411720 5165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.406516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:35989 (local address 127.0.0.1:46258)
0811 14:35:26.406808 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:26.406822 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:26.406927 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:26.407606 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:26.407607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:26.407609 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:26.407673 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:26.407674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.408773 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.408774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:26.410358 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:26.410363 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.411281 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.411283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:26.411395 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.411412 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 14:35:26.411549 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 14:35:26.411616 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":65,"thread_start_us":163,"threads_started":1}
I20260811 14:35:26.411896 5171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.406747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:43551 (local address 127.0.0.1:52266)
0811 14:35:26.407055 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:26.407065 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:26.407254 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:26.407982 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:26.407983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:26.407986 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:26.408030 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:26.408031 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.409401 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.409402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:26.410400 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:26.410404 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.411595 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.411596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:26.411667 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.411670 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:35:26.411706 (+ 36us) client_negotiation.cc:240] Negotiation successful
0811 14:35:26.411766 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":71,"thread_start_us":78,"threads_started":1}
I20260811 14:35:26.412281 5173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.406965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52266 (local address 127.2.103.67:43551)
0811 14:35:26.407745 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:26.407750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:26.407761 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:35:26.407802 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:26.407804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:26.407820 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:26.407976 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:26.408111 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.409286 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.410517 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.411463 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.412010 (+ 547us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.412106 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:26.412138 (+ 32us) server_negotiation.cc:299] Negotiation successful
0811 14:35:26.412180 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":731,"thread_start_us":129,"threads_started":1}
I20260811 14:35:26.412669 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.406440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:45653 (local address 127.0.0.1:33190)
0811 14:35:26.406765 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:26.406819 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:26.406916 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:26.407378 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:26.407381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:26.407384 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:26.407499 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:26.407502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.409130 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.409131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:26.410521 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:26.410523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.411587 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.411588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:26.411663 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.411668 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:35:26.411706 (+ 38us) client_negotiation.cc:240] Negotiation successful
0811 14:35:26.411766 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":36,"thread_start_us":107,"threads_started":1}
I20260811 14:35:26.413179 5169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.406576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33190 (local address 127.2.103.66:45653)
0811 14:35:26.407090 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:26.407097 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:26.407109 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:35:26.407223 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:26.407226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:26.407245 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:26.407594 (+ 349us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:26.407662 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.409019 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.410655 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.411463 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.412897 (+ 1434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.412971 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:26.413001 (+ 30us) server_negotiation.cc:299] Negotiation successful
0811 14:35:26.413059 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":140,"threads_started":1}
I20260811 14:35:26.413192 5035 tablet_service.cc:1467] Processing CreateTablet for tablet b3649c78200c4d6cab43300d6adc7610 (DEFAULT_TABLE table=test-table [id=3d65b27de0a3460d80d9fdc54c952c83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:26.413152 4581 tablet_service.cc:1467] Processing CreateTablet for tablet b3649c78200c4d6cab43300d6adc7610 (DEFAULT_TABLE table=test-table [id=3d65b27de0a3460d80d9fdc54c952c83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:26.413645 4581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3649c78200c4d6cab43300d6adc7610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:26.413652 5035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3649c78200c4d6cab43300d6adc7610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:26.413818 4807 tablet_service.cc:1467] Processing CreateTablet for tablet b3649c78200c4d6cab43300d6adc7610 (DEFAULT_TABLE table=test-table [id=3d65b27de0a3460d80d9fdc54c952c83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:26.414178 4807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3649c78200c4d6cab43300d6adc7610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:26.417306 5175 tablet_bootstrap.cc:492] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607: Bootstrap starting.
I20260811 14:35:26.417821 5178 tablet_bootstrap.cc:492] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f: Bootstrap starting.
I20260811 14:35:26.418198 5175 tablet_bootstrap.cc:654] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:26.418604 5175 log.cc:824] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:26.418834 5180 tablet_bootstrap.cc:492] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52: Bootstrap starting.
I20260811 14:35:26.419066 5178 tablet_bootstrap.cc:654] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:26.419507 5178 log.cc:824] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:26.419675 5180 tablet_bootstrap.cc:654] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:26.419850 5175 tablet_bootstrap.cc:492] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607: No bootstrap required, opened a new log
I20260811 14:35:26.419941 5175 ts_tablet_manager.cc:1397] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 14:35:26.420079 5180 log.cc:824] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:26.420470 5178 tablet_bootstrap.cc:492] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f: No bootstrap required, opened a new log
I20260811 14:35:26.420560 5178 ts_tablet_manager.cc:1397] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 14:35:26.421147 5180 tablet_bootstrap.cc:492] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52: No bootstrap required, opened a new log
I20260811 14:35:26.421232 5180 ts_tablet_manager.cc:1397] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 14:35:26.422302 5178 raft_consensus.cc:348] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } } peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } }
I20260811 14:35:26.422497 5178 raft_consensus.cc:374] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:26.422549 5178 raft_consensus.cc:729] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8078449ee3942d8b30c8d2b0152c43f, State: Initialized, Role: FOLLOWER
I20260811 14:35:26.422715 5178 consensus_queue.cc:260] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } } peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } }
I20260811 14:35:26.422775 5175 raft_consensus.cc:348] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } } peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } }
I20260811 14:35:26.422963 5175 raft_consensus.cc:374] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:26.423019 5175 raft_consensus.cc:729] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb2428b7f8664aa6ab5db6e748e0d607, State: Initialized, Role: FOLLOWER
I20260811 14:35:26.423081 5178 ts_tablet_manager.cc:1428] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 14:35:26.423231 5142 heartbeater.cc:503] Master 127.2.103.126:33913 was elected leader, sending a full tablet report...
I20260811 14:35:26.423197 5175 consensus_queue.cc:260] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } } peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } }
I20260811 14:35:26.423527 4693 heartbeater.cc:503] Master 127.2.103.126:33913 was elected leader, sending a full tablet report...
I20260811 14:35:26.423601 5175 ts_tablet_manager.cc:1428] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 14:35:26.423606 5180 raft_consensus.cc:348] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } } peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } }
I20260811 14:35:26.423764 5180 raft_consensus.cc:374] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:26.423807 5180 raft_consensus.cc:729] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ca1dae499ef45ab957f06a7c2808f52, State: Initialized, Role: FOLLOWER
I20260811 14:35:26.423943 5180 consensus_queue.cc:260] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } } peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } }
I20260811 14:35:26.424254 5180 ts_tablet_manager.cc:1428] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260811 14:35:26.424316 4917 heartbeater.cc:503] Master 127.2.103.126:33913 was elected leader, sending a full tablet report...
I20260811 14:35:26.478034 5188 raft_consensus.cc:482] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:35:26.478232 5188 raft_consensus.cc:504] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } } peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } }
I20260811 14:35:26.478735 5188 leader_election.cc:288] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ca1dae499ef45ab957f06a7c2808f52 (127.2.103.66:45653), cb2428b7f8664aa6ab5db6e748e0d607 (127.2.103.65:35989)
I20260811 14:35:26.482173 5170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.479056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:45575 (local address 127.2.103.65:35989)
0811 14:35:26.479106 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:26.479108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:26.479124 (+ 16us) server_negotiation.cc:407] Connection header received
0811 14:35:26.479267 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:26.479269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:26.479278 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:26.479365 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:26.479555 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.480104 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.481272 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.481716 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.481821 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.481831 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:26.482028 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 14:35:26.482077 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 14:35:26.482199 5169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.479134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:47379 (local address 127.2.103.66:45653)
0811 14:35:26.479194 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:26.479196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:26.479347 (+ 151us) server_negotiation.cc:407] Connection header received
0811 14:35:26.479435 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:26.479436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:26.479449 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:26.479529 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:26.479720 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.480234 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.481326 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.481749 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.481851 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.481862 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:26.482068 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 14:35:26.482116 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 14:35:26.482215 4973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.479004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:35989 (local address 127.2.103.67:45575)
0811 14:35:26.479070 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:26.479084 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:26.479140 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:26.479376 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:26.479378 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:26.479380 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:26.479463 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:26.479464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.480256 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.480257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:26.481127 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:26.481129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.481821 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.481822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:26.481931 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.481936 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:35:26.482020 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 14:35:26.482071 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 14:35:26.482234 5193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.479039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:45653 (local address 127.2.103.67:47379)
0811 14:35:26.479248 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:26.479257 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:26.479356 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:26.479536 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:26.479537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:26.479539 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:26.479636 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:26.479638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.480346 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.480347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:26.481190 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:26.481193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.481853 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.481854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:26.481970 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.481974 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:35:26.482060 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 14:35:26.482109 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":138,"threads_started":1}
I20260811 14:35:26.482642 4613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3649c78200c4d6cab43300d6adc7610" candidate_uuid: "d8078449ee3942d8b30c8d2b0152c43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" is_pre_election: true
I20260811 14:35:26.482851 4613 raft_consensus.cc:2454] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8078449ee3942d8b30c8d2b0152c43f in term 0.
I20260811 14:35:26.482782 4837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3649c78200c4d6cab43300d6adc7610" candidate_uuid: "d8078449ee3942d8b30c8d2b0152c43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ca1dae499ef45ab957f06a7c2808f52" is_pre_election: true
I20260811 14:35:26.482995 4837 raft_consensus.cc:2454] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8078449ee3942d8b30c8d2b0152c43f in term 0.
I20260811 14:35:26.483244 4961 leader_election.cc:302] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ca1dae499ef45ab957f06a7c2808f52, d8078449ee3942d8b30c8d2b0152c43f; no voters:
I20260811 14:35:26.483466 5188 raft_consensus.cc:2781] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:35:26.483577 5188 raft_consensus.cc:482] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:35:26.483620 5188 raft_consensus.cc:3037] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:26.484704 5188 raft_consensus.cc:504] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } } peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } }
I20260811 14:35:26.484890 5188 leader_election.cc:288] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [CANDIDATE]: Term 1 election: Requested vote from peers 0ca1dae499ef45ab957f06a7c2808f52 (127.2.103.66:45653), cb2428b7f8664aa6ab5db6e748e0d607 (127.2.103.65:35989)
I20260811 14:35:26.485102 4613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3649c78200c4d6cab43300d6adc7610" candidate_uuid: "d8078449ee3942d8b30c8d2b0152c43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb2428b7f8664aa6ab5db6e748e0d607"
I20260811 14:35:26.485126 4837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3649c78200c4d6cab43300d6adc7610" candidate_uuid: "d8078449ee3942d8b30c8d2b0152c43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ca1dae499ef45ab957f06a7c2808f52"
I20260811 14:35:26.485201 4613 raft_consensus.cc:3037] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:26.485206 4837 raft_consensus.cc:3037] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52 [term 0 FOLLOWER]: Advancing to term 1
W20260811 14:35:26.485236 4694 tablet.cc:2369] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:35:26.486219 4837 raft_consensus.cc:2454] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8078449ee3942d8b30c8d2b0152c43f in term 1.
I20260811 14:35:26.486217 4613 raft_consensus.cc:2454] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8078449ee3942d8b30c8d2b0152c43f in term 1.
I20260811 14:35:26.486469 4961 leader_election.cc:302] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ca1dae499ef45ab957f06a7c2808f52, d8078449ee3942d8b30c8d2b0152c43f; no voters:
I20260811 14:35:26.486660 5188 raft_consensus.cc:2781] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:26.486948 5188 raft_consensus.cc:686] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [term 1 LEADER]: Becoming Leader. State: Replica: d8078449ee3942d8b30c8d2b0152c43f, State: Running, Role: LEADER
I20260811 14:35:26.487090 5188 consensus_queue.cc:237] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } } peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } }
I20260811 14:35:26.488160 4347 catalog_manager.cc:5495] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f reported cstate change: term changed from 0 to 1, leader changed from <none> to d8078449ee3942d8b30c8d2b0152c43f (127.2.103.67). New cstate: current_term: 1 leader_uuid: "d8078449ee3942d8b30c8d2b0152c43f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } health_report { overall_health: UNKNOWN } } }
I20260811 14:35:26.511197 5173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.508044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52272 (local address 127.2.103.67:43551)
0811 14:35:26.508125 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:26.508127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:26.508145 (+ 18us) server_negotiation.cc:407] Connection header received
0811 14:35:26.508187 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:26.508187 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:26.508207 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:26.508285 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:26.508536 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.509113 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.510001 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.510208 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.510273 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.510707 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:26.510828 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:26.510914 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:26.511003 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 14:35:26.511030 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 14:35:26.513962 4613 raft_consensus.cc:1261] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607 [term 1 FOLLOWER]: Refusing update from remote peer d8078449ee3942d8b30c8d2b0152c43f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:26.514353 5188 consensus_queue.cc:1035] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:26.515789 4837 raft_consensus.cc:1261] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52 [term 1 FOLLOWER]: Refusing update from remote peer d8078449ee3942d8b30c8d2b0152c43f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:26.516140 5195 consensus_queue.cc:1035] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:26.517431 5201 mvcc.cc:205] Tried to move back new op lower bound from 7317335762997088256 to 7317335762891812864. Current Snapshot: MvccSnapshot[applied={T|T < 7317335762997088256}]
I20260811 14:35:26.517875 5202 mvcc.cc:205] Tried to move back new op lower bound from 7317335762997088256 to 7317335762891812864. Current Snapshot: MvccSnapshot[applied={T|T < 7317335762997088256}]
I20260811 14:35:26.518877 5206 mvcc.cc:205] Tried to move back new op lower bound from 7317335762997088256 to 7317335762891812864. Current Snapshot: MvccSnapshot[applied={T|T < 7317335762997088256}]
I20260811 14:35:26.523528 4347 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:26.523705 4347 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:26.524878 4347 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68: Sending DeleteTablet for 3 replicas of tablet b3649c78200c4d6cab43300d6adc7610
I20260811 14:35:26.525494 4807 tablet_service.cc:1514] Processing DeleteTablet for tablet b3649c78200c4d6cab43300d6adc7610 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:26 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:33190
I20260811 14:35:26.525487 5035 tablet_service.cc:1514] Processing DeleteTablet for tablet b3649c78200c4d6cab43300d6adc7610 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:26 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:52266
I20260811 14:35:26.525480 4581 tablet_service.cc:1514] Processing DeleteTablet for tablet b3649c78200c4d6cab43300d6adc7610 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:26 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:46258
I20260811 14:35:26.525768 5216 tablet_replica.cc:330] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607: stopping tablet replica
I20260811 14:35:26.525899 5216 raft_consensus.cc:2229] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:26.526074 5216 raft_consensus.cc:2258] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:26.526162 5215 tablet_replica.cc:330] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f: stopping tablet replica
I20260811 14:35:26.526284 5215 raft_consensus.cc:2229] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:35:26.526226 4347 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 14:35:26.526392 5215 raft_consensus.cc:2258] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 14:35:26.526412 4347 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:35:26.526546 5214 tablet_replica.cc:330] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52: stopping tablet replica
I20260811 14:35:26.526749 5216 ts_tablet_manager.cc:1905] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:26.526680 5214 raft_consensus.cc:2229] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:26.527400 5214 raft_consensus.cc:2258] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:26.527912 5215 ts_tablet_manager.cc:1905] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:26.527949 4581 tablet_service.cc:1467] Processing CreateTablet for tablet 3c99fecf0a72469b83a8fdcc35d8c1f3 (DEFAULT_TABLE table=test-table [id=410e0456a8fb4fbda19f68e7248c0ea6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:26.528074 4581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c99fecf0a72469b83a8fdcc35d8c1f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:26.528955 5035 tablet_service.cc:1467] Processing CreateTablet for tablet 3c99fecf0a72469b83a8fdcc35d8c1f3 (DEFAULT_TABLE table=test-table [id=410e0456a8fb4fbda19f68e7248c0ea6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:26.529384 4807 tablet_service.cc:1467] Processing CreateTablet for tablet 3c99fecf0a72469b83a8fdcc35d8c1f3 (DEFAULT_TABLE table=test-table [id=410e0456a8fb4fbda19f68e7248c0ea6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:26.529502 5214 ts_tablet_manager.cc:1905] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:26.529582 4807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c99fecf0a72469b83a8fdcc35d8c1f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:26.529800 5035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c99fecf0a72469b83a8fdcc35d8c1f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:26.531250 5178 tablet_bootstrap.cc:492] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f: Bootstrap starting.
I20260811 14:35:26.531344 5180 tablet_bootstrap.cc:492] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52: Bootstrap starting.
I20260811 14:35:26.531730 5178 tablet_bootstrap.cc:654] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:26.531829 5180 tablet_bootstrap.cc:654] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:26.532125 5214 ts_tablet_manager.cc:1918] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:35:26.532305 5214 log.cc:1196] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/b3649c78200c4d6cab43300d6adc7610
I20260811 14:35:26.532979 5214 ts_tablet_manager.cc:1939] T b3649c78200c4d6cab43300d6adc7610 P 0ca1dae499ef45ab957f06a7c2808f52: Deleting consensus metadata
I20260811 14:35:26.533010 5215 ts_tablet_manager.cc:1918] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:35:26.533089 5215 log.cc:1196] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/b3649c78200c4d6cab43300d6adc7610
I20260811 14:35:26.533375 5175 tablet_bootstrap.cc:492] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P cb2428b7f8664aa6ab5db6e748e0d607: Bootstrap starting.
I20260811 14:35:26.533401 5215 ts_tablet_manager.cc:1939] T b3649c78200c4d6cab43300d6adc7610 P d8078449ee3942d8b30c8d2b0152c43f: Deleting consensus metadata
I20260811 14:35:26.533850 5175 tablet_bootstrap.cc:654] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P cb2428b7f8664aa6ab5db6e748e0d607: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:26.533871 5216 ts_tablet_manager.cc:1918] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:35:26.533939 5216 log.cc:1196] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-0/wal/wals/b3649c78200c4d6cab43300d6adc7610
I20260811 14:35:26.533896 4325 catalog_manager.cc:4841] TS d8078449ee3942d8b30c8d2b0152c43f (127.2.103.67:43551): tablet b3649c78200c4d6cab43300d6adc7610 (table test-table [id=3d65b27de0a3460d80d9fdc54c952c83]) successfully deleted
I20260811 14:35:26.534231 5216 ts_tablet_manager.cc:1939] T b3649c78200c4d6cab43300d6adc7610 P cb2428b7f8664aa6ab5db6e748e0d607: Deleting consensus metadata
I20260811 14:35:26.534433 5178 tablet_bootstrap.cc:492] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f: No bootstrap required, opened a new log
I20260811 14:35:26.534488 5178 ts_tablet_manager.cc:1397] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260811 14:35:26.534703 4326 catalog_manager.cc:4841] TS cb2428b7f8664aa6ab5db6e748e0d607 (127.2.103.65:35989): tablet b3649c78200c4d6cab43300d6adc7610 (table test-table [id=3d65b27de0a3460d80d9fdc54c952c83]) successfully deleted
I20260811 14:35:26.534683 5178 raft_consensus.cc:348] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } }
I20260811 14:35:26.534770 5178 raft_consensus.cc:374] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:26.534765 4325 catalog_manager.cc:4841] TS 0ca1dae499ef45ab957f06a7c2808f52 (127.2.103.66:45653): tablet b3649c78200c4d6cab43300d6adc7610 (table test-table [id=3d65b27de0a3460d80d9fdc54c952c83]) successfully deleted
I20260811 14:35:26.534793 5178 raft_consensus.cc:729] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8078449ee3942d8b30c8d2b0152c43f, State: Initialized, Role: FOLLOWER
I20260811 14:35:26.534849 5178 consensus_queue.cc:260] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } }
I20260811 14:35:26.534940 5178 ts_tablet_manager.cc:1428] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:26.535205 5180 tablet_bootstrap.cc:492] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52: No bootstrap required, opened a new log
I20260811 14:35:26.535259 5180 ts_tablet_manager.cc:1397] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20260811 14:35:26.535353 5175 tablet_bootstrap.cc:492] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P cb2428b7f8664aa6ab5db6e748e0d607: No bootstrap required, opened a new log
I20260811 14:35:26.535389 5175 ts_tablet_manager.cc:1397] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P cb2428b7f8664aa6ab5db6e748e0d607: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:35:26.535434 5180 raft_consensus.cc:348] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } }
I20260811 14:35:26.535514 5180 raft_consensus.cc:374] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:26.535534 5180 raft_consensus.cc:729] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ca1dae499ef45ab957f06a7c2808f52, State: Initialized, Role: FOLLOWER
I20260811 14:35:26.535585 5180 consensus_queue.cc:260] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } }
I20260811 14:35:26.535676 5180 ts_tablet_manager.cc:1428] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:26.535734 5175 raft_consensus.cc:348] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P cb2428b7f8664aa6ab5db6e748e0d607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } }
I20260811 14:35:26.535799 5175 raft_consensus.cc:374] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P cb2428b7f8664aa6ab5db6e748e0d607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:26.535817 5175 raft_consensus.cc:729] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P cb2428b7f8664aa6ab5db6e748e0d607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb2428b7f8664aa6ab5db6e748e0d607, State: Initialized, Role: FOLLOWER
I20260811 14:35:26.535857 5175 consensus_queue.cc:260] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P cb2428b7f8664aa6ab5db6e748e0d607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } }
I20260811 14:35:26.535969 5175 ts_tablet_manager.cc:1428] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P cb2428b7f8664aa6ab5db6e748e0d607: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260811 14:35:26.610740 5144 tablet.cc:2369] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:35:26.673830 4918 tablet.cc:2369] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:35:26.941563 5209 raft_consensus.cc:482] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:35:26.941730 5209 raft_consensus.cc:504] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } }
I20260811 14:35:26.942025 5209 leader_election.cc:288] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb2428b7f8664aa6ab5db6e748e0d607 (127.2.103.65:35989), 0ca1dae499ef45ab957f06a7c2808f52 (127.2.103.66:45653)
I20260811 14:35:26.942346 4837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c99fecf0a72469b83a8fdcc35d8c1f3" candidate_uuid: "d8078449ee3942d8b30c8d2b0152c43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ca1dae499ef45ab957f06a7c2808f52" is_pre_election: true
I20260811 14:35:26.942361 4613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c99fecf0a72469b83a8fdcc35d8c1f3" candidate_uuid: "d8078449ee3942d8b30c8d2b0152c43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" is_pre_election: true
I20260811 14:35:26.942451 4837 raft_consensus.cc:2454] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8078449ee3942d8b30c8d2b0152c43f in term 0.
I20260811 14:35:26.942476 4613 raft_consensus.cc:2454] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P cb2428b7f8664aa6ab5db6e748e0d607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8078449ee3942d8b30c8d2b0152c43f in term 0.
I20260811 14:35:26.942654 4961 leader_election.cc:302] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ca1dae499ef45ab957f06a7c2808f52, d8078449ee3942d8b30c8d2b0152c43f; no voters:
I20260811 14:35:26.942788 5209 raft_consensus.cc:2781] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:35:26.942848 5209 raft_consensus.cc:482] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:35:26.942869 5209 raft_consensus.cc:3037] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:26.943656 5209 raft_consensus.cc:504] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } }
I20260811 14:35:26.943811 5209 leader_election.cc:288] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [CANDIDATE]: Term 1 election: Requested vote from peers cb2428b7f8664aa6ab5db6e748e0d607 (127.2.103.65:35989), 0ca1dae499ef45ab957f06a7c2808f52 (127.2.103.66:45653)
I20260811 14:35:26.944046 4837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c99fecf0a72469b83a8fdcc35d8c1f3" candidate_uuid: "d8078449ee3942d8b30c8d2b0152c43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ca1dae499ef45ab957f06a7c2808f52"
I20260811 14:35:26.944129 4837 raft_consensus.cc:3037] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:26.944068 4613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c99fecf0a72469b83a8fdcc35d8c1f3" candidate_uuid: "d8078449ee3942d8b30c8d2b0152c43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb2428b7f8664aa6ab5db6e748e0d607"
I20260811 14:35:26.944157 4613 raft_consensus.cc:3037] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P cb2428b7f8664aa6ab5db6e748e0d607 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:26.944774 4837 raft_consensus.cc:2454] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8078449ee3942d8b30c8d2b0152c43f in term 1.
I20260811 14:35:26.944892 4613 raft_consensus.cc:2454] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P cb2428b7f8664aa6ab5db6e748e0d607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8078449ee3942d8b30c8d2b0152c43f in term 1.
I20260811 14:35:26.944979 4961 leader_election.cc:302] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ca1dae499ef45ab957f06a7c2808f52, d8078449ee3942d8b30c8d2b0152c43f; no voters:
I20260811 14:35:26.945070 5209 raft_consensus.cc:2781] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:26.945151 5209 raft_consensus.cc:686] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [term 1 LEADER]: Becoming Leader. State: Replica: d8078449ee3942d8b30c8d2b0152c43f, State: Running, Role: LEADER
I20260811 14:35:26.945205 5209 consensus_queue.cc:237] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } }
I20260811 14:35:26.945840 4347 catalog_manager.cc:5495] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f reported cstate change: term changed from 0 to 1, leader changed from <none> to d8078449ee3942d8b30c8d2b0152c43f (127.2.103.67). New cstate: current_term: 1 leader_uuid: "d8078449ee3942d8b30c8d2b0152c43f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } health_report { overall_health: HEALTHY } } }
Aug 11 14:35:26 dist-test-slave-twwt krb5kdc[4284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458925, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:27.003499 5235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.995531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48808 (local address 127.2.103.126:33913)
0811 14:35:26.995678 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:26.995681 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:26.995779 (+ 98us) server_negotiation.cc:407] Connection header received
0811 14:35:26.995888 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:26.995889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:26.995906 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:26.996008 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:26.996576 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.997113 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.997865 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.998068 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.998161 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:27.000412 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:27.000413 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:27.000413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:27.000422 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:27.001999 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:27.002496 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:27.002498 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:27.002498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:27.002565 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:27.002826 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:27.002827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:27.002828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:27.002955 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:27.003050 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:27.003210 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 14:35:27.003359 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260811 14:35:27.003553 5234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:26.995427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:33913 (local address 127.0.0.1:48808)
0811 14:35:26.995688 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:26.995700 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:26.995801 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:26.996028 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:26.996029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:26.996397 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:26.996475 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:26.996478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.997210 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.997211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:26.997768 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:26.997769 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:26.998162 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:26.998163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:26.998214 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:26.998853 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:26.998858 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:27.000311 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:27.002134 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:27.002144 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:27.002148 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:27.002392 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:27.002679 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:27.002680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:27.002680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:27.002754 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:27.003035 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:27.003045 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:27.003096 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 14:35:27.003186 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 14:35:27.003420 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":115,"thread_start_us":149,"threads_started":1}
I20260811 14:35:27.006120 4347 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:48808:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 14:35:27.006497 4347 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:35:27.008301 5035 tablet_service.cc:1467] Processing CreateTablet for tablet dbce472184954010b4ce63e439239d36 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a8535b5cc16547b6b8edadd6ee49e229]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 14:35:27.008267 4581 tablet_service.cc:1467] Processing CreateTablet for tablet dbce472184954010b4ce63e439239d36 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a8535b5cc16547b6b8edadd6ee49e229]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 14:35:27.008306 4807 tablet_service.cc:1467] Processing CreateTablet for tablet dbce472184954010b4ce63e439239d36 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a8535b5cc16547b6b8edadd6ee49e229]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 14:35:27.008531 4581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbce472184954010b4ce63e439239d36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:27.008530 4807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbce472184954010b4ce63e439239d36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:27.008531 5035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbce472184954010b4ce63e439239d36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:27.009902 5178 tablet_bootstrap.cc:492] T dbce472184954010b4ce63e439239d36 P d8078449ee3942d8b30c8d2b0152c43f: Bootstrap starting.
I20260811 14:35:27.010303 5180 tablet_bootstrap.cc:492] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52: Bootstrap starting.
I20260811 14:35:27.010335 5175 tablet_bootstrap.cc:492] T dbce472184954010b4ce63e439239d36 P cb2428b7f8664aa6ab5db6e748e0d607: Bootstrap starting.
I20260811 14:35:27.010507 5178 tablet_bootstrap.cc:654] T dbce472184954010b4ce63e439239d36 P d8078449ee3942d8b30c8d2b0152c43f: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:27.010701 5175 tablet_bootstrap.cc:654] T dbce472184954010b4ce63e439239d36 P cb2428b7f8664aa6ab5db6e748e0d607: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:27.010783 5180 tablet_bootstrap.cc:654] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:27.011255 5178 tablet_bootstrap.cc:492] T dbce472184954010b4ce63e439239d36 P d8078449ee3942d8b30c8d2b0152c43f: No bootstrap required, opened a new log
I20260811 14:35:27.011310 5178 ts_tablet_manager.cc:1397] T dbce472184954010b4ce63e439239d36 P d8078449ee3942d8b30c8d2b0152c43f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:27.011457 5178 raft_consensus.cc:348] T dbce472184954010b4ce63e439239d36 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } }
I20260811 14:35:27.011505 5175 tablet_bootstrap.cc:492] T dbce472184954010b4ce63e439239d36 P cb2428b7f8664aa6ab5db6e748e0d607: No bootstrap required, opened a new log
I20260811 14:35:27.011505 5180 tablet_bootstrap.cc:492] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52: No bootstrap required, opened a new log
I20260811 14:35:27.011535 5178 raft_consensus.cc:374] T dbce472184954010b4ce63e439239d36 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:27.011549 5180 ts_tablet_manager.cc:1397] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:27.011549 5175 ts_tablet_manager.cc:1397] T dbce472184954010b4ce63e439239d36 P cb2428b7f8664aa6ab5db6e748e0d607: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 14:35:27.011556 5178 raft_consensus.cc:729] T dbce472184954010b4ce63e439239d36 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8078449ee3942d8b30c8d2b0152c43f, State: Initialized, Role: FOLLOWER
I20260811 14:35:27.011633 5178 consensus_queue.cc:260] T dbce472184954010b4ce63e439239d36 P d8078449ee3942d8b30c8d2b0152c43f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } }
I20260811 14:35:27.011840 5180 raft_consensus.cc:348] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } }
I20260811 14:35:27.011919 5180 raft_consensus.cc:374] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:27.011940 5180 raft_consensus.cc:729] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ca1dae499ef45ab957f06a7c2808f52, State: Initialized, Role: FOLLOWER
I20260811 14:35:27.011981 5180 consensus_queue.cc:260] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } }
I20260811 14:35:27.011746 5209 tablet_replica.cc:439] T dbce472184954010b4ce63e439239d36 P d8078449ee3942d8b30c8d2b0152c43f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } } }
I20260811 14:35:27.012005 5175 raft_consensus.cc:348] T dbce472184954010b4ce63e439239d36 P cb2428b7f8664aa6ab5db6e748e0d607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } }
I20260811 14:35:27.012053 5209 tablet_replica.cc:442] T dbce472184954010b4ce63e439239d36 P d8078449ee3942d8b30c8d2b0152c43f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:35:27.012072 5175 raft_consensus.cc:374] T dbce472184954010b4ce63e439239d36 P cb2428b7f8664aa6ab5db6e748e0d607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:27.012081 5180 ts_tablet_manager.cc:1428] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:27.012099 5175 raft_consensus.cc:729] T dbce472184954010b4ce63e439239d36 P cb2428b7f8664aa6ab5db6e748e0d607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb2428b7f8664aa6ab5db6e748e0d607, State: Initialized, Role: FOLLOWER
I20260811 14:35:27.012141 5175 consensus_queue.cc:260] T dbce472184954010b4ce63e439239d36 P cb2428b7f8664aa6ab5db6e748e0d607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } }
I20260811 14:35:27.012209 5178 ts_tablet_manager.cc:1428] T dbce472184954010b4ce63e439239d36 P d8078449ee3942d8b30c8d2b0152c43f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 14:35:27.012226 5175 ts_tablet_manager.cc:1428] T dbce472184954010b4ce63e439239d36 P cb2428b7f8664aa6ab5db6e748e0d607: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 14:35:27.012193 5191 tablet_replica.cc:439] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } } }
I20260811 14:35:27.012301 5191 tablet_replica.cc:442] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:35:27.012400 5189 tablet_replica.cc:439] T dbce472184954010b4ce63e439239d36 P cb2428b7f8664aa6ab5db6e748e0d607: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } } }
I20260811 14:35:27.012498 5189 tablet_replica.cc:442] T dbce472184954010b4ce63e439239d36 P cb2428b7f8664aa6ab5db6e748e0d607: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:35:27.027580 5191 raft_consensus.cc:482] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:35:27.027679 5191 raft_consensus.cc:504] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } }
I20260811 14:35:27.028147 5191 leader_election.cc:288] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb2428b7f8664aa6ab5db6e748e0d607 (127.2.103.65:35989), d8078449ee3942d8b30c8d2b0152c43f (127.2.103.67:43551)
I20260811 14:35:27.031378 5243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:27.028306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:35989 (local address 127.2.103.66:58755)
0811 14:35:27.028443 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:27.028459 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:27.028523 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:27.028737 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:27.028739 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:27.028741 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:27.028842 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:27.028844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.029595 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.029595 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:27.030416 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:27.030419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.031109 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.031110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:27.031180 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:27.031183 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:35:27.031266 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 14:35:27.031306 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260811 14:35:27.031378 5244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:27.028398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:32805 (local address 127.2.103.67:43551)
0811 14:35:27.028557 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:27.028560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:27.028568 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:35:27.028595 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:27.028596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:27.028607 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:27.028679 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:27.028856 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.029371 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.030448 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.030965 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.031027 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:27.031036 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:27.031241 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 14:35:27.031293 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":87,"threads_started":1}
I20260811 14:35:27.031409 5245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:27.028440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:58755 (local address 127.2.103.65:35989)
0811 14:35:27.028570 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:27.028572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:27.028581 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:35:27.028607 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:27.028608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:27.028618 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:27.028714 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:27.028926 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.029510 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.030508 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.031017 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.031124 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:27.031134 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:27.031283 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 14:35:27.031329 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":54,"threads_started":1}
I20260811 14:35:27.031409 5241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:27.028322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:43551 (local address 127.2.103.66:32805)
0811 14:35:27.028453 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:27.028465 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:27.028522 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:27.028712 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:27.028713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:27.028715 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:27.028782 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:27.028784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.029457 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.029458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:27.030316 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:27.030319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.031086 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.031088 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:27.031156 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:27.031161 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:35:27.031210 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 14:35:27.031271 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":54,"threads_started":1}
I20260811 14:35:27.031679 4613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbce472184954010b4ce63e439239d36" candidate_uuid: "0ca1dae499ef45ab957f06a7c2808f52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" is_pre_election: true
I20260811 14:35:27.031791 4613 raft_consensus.cc:2454] T dbce472184954010b4ce63e439239d36 P cb2428b7f8664aa6ab5db6e748e0d607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ca1dae499ef45ab957f06a7c2808f52 in term 0.
I20260811 14:35:27.031888 5072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbce472184954010b4ce63e439239d36" candidate_uuid: "0ca1dae499ef45ab957f06a7c2808f52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8078449ee3942d8b30c8d2b0152c43f" is_pre_election: true
I20260811 14:35:27.031973 4725 leader_election.cc:302] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ca1dae499ef45ab957f06a7c2808f52, cb2428b7f8664aa6ab5db6e748e0d607; no voters:
I20260811 14:35:27.032042 5072 raft_consensus.cc:2454] T dbce472184954010b4ce63e439239d36 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ca1dae499ef45ab957f06a7c2808f52 in term 0.
I20260811 14:35:27.032151 5191 raft_consensus.cc:2781] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:35:27.032200 5191 raft_consensus.cc:482] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:35:27.032223 5191 raft_consensus.cc:3037] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:27.033072 5191 raft_consensus.cc:504] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } }
I20260811 14:35:27.033247 5191 leader_election.cc:288] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52 [CANDIDATE]: Term 1 election: Requested vote from peers cb2428b7f8664aa6ab5db6e748e0d607 (127.2.103.65:35989), d8078449ee3942d8b30c8d2b0152c43f (127.2.103.67:43551)
I20260811 14:35:27.033499 4613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbce472184954010b4ce63e439239d36" candidate_uuid: "0ca1dae499ef45ab957f06a7c2808f52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb2428b7f8664aa6ab5db6e748e0d607"
I20260811 14:35:27.033612 4613 raft_consensus.cc:3037] T dbce472184954010b4ce63e439239d36 P cb2428b7f8664aa6ab5db6e748e0d607 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:27.033633 5072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbce472184954010b4ce63e439239d36" candidate_uuid: "0ca1dae499ef45ab957f06a7c2808f52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8078449ee3942d8b30c8d2b0152c43f"
I20260811 14:35:27.033725 5072 raft_consensus.cc:3037] T dbce472184954010b4ce63e439239d36 P d8078449ee3942d8b30c8d2b0152c43f [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:27.034385 4613 raft_consensus.cc:2454] T dbce472184954010b4ce63e439239d36 P cb2428b7f8664aa6ab5db6e748e0d607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ca1dae499ef45ab957f06a7c2808f52 in term 1.
I20260811 14:35:27.034485 5072 raft_consensus.cc:2454] T dbce472184954010b4ce63e439239d36 P d8078449ee3942d8b30c8d2b0152c43f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ca1dae499ef45ab957f06a7c2808f52 in term 1.
I20260811 14:35:27.034585 4725 leader_election.cc:302] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ca1dae499ef45ab957f06a7c2808f52, cb2428b7f8664aa6ab5db6e748e0d607; no voters:
I20260811 14:35:27.034759 5191 raft_consensus.cc:2781] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:27.034955 5191 raft_consensus.cc:686] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52 [term 1 LEADER]: Becoming Leader. State: Replica: 0ca1dae499ef45ab957f06a7c2808f52, State: Running, Role: LEADER
I20260811 14:35:27.035041 5191 consensus_queue.cc:237] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } }
I20260811 14:35:27.035494 5246 tablet_replica.cc:439] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52: TxnStatusTablet state changed. Reason: New leader 0ca1dae499ef45ab957f06a7c2808f52. Latest consensus state: current_term: 1 leader_uuid: "0ca1dae499ef45ab957f06a7c2808f52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } } }
I20260811 14:35:27.035594 5246 tablet_replica.cc:442] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52: This TxnStatusTablet replica's current role is: LEADER
I20260811 14:35:27.035789 5250 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 14:35:27.035938 5250 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 14:35:27.035907 4347 catalog_manager.cc:5495] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ca1dae499ef45ab957f06a7c2808f52 (127.2.103.66). New cstate: current_term: 1 leader_uuid: "0ca1dae499ef45ab957f06a7c2808f52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } health_report { overall_health: UNKNOWN } } }
I20260811 14:35:27.046155 5255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:27.042709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33198 (local address 127.2.103.66:45653)
0811 14:35:27.042905 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:27.042912 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:27.042920 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:35:27.042962 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:27.042963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:27.042998 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:27.043179 (+ 181us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:27.043359 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.043938 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.045140 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.045400 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.045490 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:27.045754 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:27.045886 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:27.045958 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:27.046059 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 14:35:27.046080 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":151,"threads_started":1}
I20260811 14:35:27.046206 5234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:27.042576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:45653 (local address 127.0.0.1:33198)
0811 14:35:27.042686 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:27.042700 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:27.042763 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:27.043212 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:27.043213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:27.043215 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:35:27.043274 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:27.043277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.044027 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.044028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:27.045003 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:27.045007 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.045495 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.045497 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:27.045590 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:27.045620 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:35:27.045979 (+ 359us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:35:27.045992 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 14:35:27.046056 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 14:35:27.046121 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 14:35:27.048049 4613 raft_consensus.cc:1261] T dbce472184954010b4ce63e439239d36 P cb2428b7f8664aa6ab5db6e748e0d607 [term 1 FOLLOWER]: Refusing update from remote peer 0ca1dae499ef45ab957f06a7c2808f52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:27.048267 5072 raft_consensus.cc:1261] T dbce472184954010b4ce63e439239d36 P d8078449ee3942d8b30c8d2b0152c43f [term 1 FOLLOWER]: Refusing update from remote peer 0ca1dae499ef45ab957f06a7c2808f52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:27.048446 5246 consensus_queue.cc:1035] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:27.048597 5191 consensus_queue.cc:1035] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:27.049263 5189 tablet_replica.cc:439] T dbce472184954010b4ce63e439239d36 P cb2428b7f8664aa6ab5db6e748e0d607: TxnStatusTablet state changed. Reason: New leader 0ca1dae499ef45ab957f06a7c2808f52. Latest consensus state: current_term: 1 leader_uuid: "0ca1dae499ef45ab957f06a7c2808f52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } } }
I20260811 14:35:27.049365 5189 tablet_replica.cc:442] T dbce472184954010b4ce63e439239d36 P cb2428b7f8664aa6ab5db6e748e0d607: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:35:27.049299 5209 tablet_replica.cc:439] T dbce472184954010b4ce63e439239d36 P d8078449ee3942d8b30c8d2b0152c43f: TxnStatusTablet state changed. Reason: New leader 0ca1dae499ef45ab957f06a7c2808f52. Latest consensus state: current_term: 1 leader_uuid: "0ca1dae499ef45ab957f06a7c2808f52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } } }
I20260811 14:35:27.049397 5209 tablet_replica.cc:442] T dbce472184954010b4ce63e439239d36 P d8078449ee3942d8b30c8d2b0152c43f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:35:27.050006 5246 tablet_replica.cc:439] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0ca1dae499ef45ab957f06a7c2808f52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } } }
I20260811 14:35:27.050104 5246 tablet_replica.cc:442] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52: This TxnStatusTablet replica's current role is: LEADER
I20260811 14:35:27.050447 5189 tablet_replica.cc:439] T dbce472184954010b4ce63e439239d36 P cb2428b7f8664aa6ab5db6e748e0d607: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0ca1dae499ef45ab957f06a7c2808f52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } } }
I20260811 14:35:27.050541 5189 tablet_replica.cc:442] T dbce472184954010b4ce63e439239d36 P cb2428b7f8664aa6ab5db6e748e0d607: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:35:27.051002 5246 tablet_replica.cc:439] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0ca1dae499ef45ab957f06a7c2808f52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } } }
I20260811 14:35:27.051119 5246 tablet_replica.cc:442] T dbce472184954010b4ce63e439239d36 P 0ca1dae499ef45ab957f06a7c2808f52: This TxnStatusTablet replica's current role is: LEADER
I20260811 14:35:27.051649 5209 tablet_replica.cc:439] T dbce472184954010b4ce63e439239d36 P d8078449ee3942d8b30c8d2b0152c43f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0ca1dae499ef45ab957f06a7c2808f52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 } } peers { permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 } } peers { permanent_uuid: "d8078449ee3942d8b30c8d2b0152c43f" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43551 } } }
I20260811 14:35:27.051748 5209 tablet_replica.cc:442] T dbce472184954010b4ce63e439239d36 P d8078449ee3942d8b30c8d2b0152c43f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:35:27.059515 5255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:27.056662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:48047 (local address 127.2.103.66:45653)
0811 14:35:27.056731 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:27.056733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:27.056745 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:35:27.056775 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:27.056776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:27.056785 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:27.056833 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:27.057007 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.057497 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.058705 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.058889 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.058943 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:27.059231 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:27.059261 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:27.059361 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:27.059413 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 14:35:27.059434 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 14:35:27.059580 5276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:27.056222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:45653 (local address 127.2.103.67:48047)
0811 14:35:27.056486 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:27.056499 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:27.056556 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:27.056861 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:27.056863 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:27.056865 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:35:27.056928 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:27.056929 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.057581 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.057582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:27.058591 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:27.058595 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.059056 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.059057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:27.059123 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:27.059142 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:35:27.059353 (+ 211us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:35:27.059355 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:35:27.059408 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 14:35:27.059465 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":164,"threads_started":1}
I20260811 14:35:27.062005 4837 raft_consensus.cc:1261] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52 [term 1 FOLLOWER]: Refusing update from remote peer d8078449ee3942d8b30c8d2b0152c43f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:27.062005 4613 raft_consensus.cc:1261] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P cb2428b7f8664aa6ab5db6e748e0d607 [term 1 FOLLOWER]: Refusing update from remote peer d8078449ee3942d8b30c8d2b0152c43f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:27.062232 5195 consensus_queue.cc:1035] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ca1dae499ef45ab957f06a7c2808f52" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:27.062363 5209 consensus_queue.cc:1035] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb2428b7f8664aa6ab5db6e748e0d607" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:27.073133 5244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:27.070161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:60291 (local address 127.2.103.67:43551)
0811 14:35:27.070205 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:27.070208 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:27.070395 (+ 187us) server_negotiation.cc:407] Connection header received
0811 14:35:27.070423 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:27.070424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:27.070435 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:27.070483 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:27.070664 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.071166 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.072345 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.072563 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.072645 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:27.072854 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:27.072894 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:27.072959 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:27.073020 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 14:35:27.073044 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 14:35:27.073189 5284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:27.070063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:43551 (local address 127.2.103.66:60291)
0811 14:35:27.070303 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:27.070315 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:27.070380 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:27.070486 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:27.070487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:27.070488 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:35:27.070578 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:27.070579 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.071281 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.071282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:27.072248 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:27.072251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.072644 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.072645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:27.072742 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:27.072758 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:35:27.072961 (+ 203us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:35:27.072964 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:35:27.073007 (+ 43us) client_negotiation.cc:240] Negotiation successful
0811 14:35:27.073056 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":112,"threads_started":1}
I20260811 14:35:27.085891 4347 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:27.086001 4347 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:27.086702 4347 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dfbd7aa767f54a0b82585f955efafe68: Sending DeleteTablet for 3 replicas of tablet 3c99fecf0a72469b83a8fdcc35d8c1f3
I20260811 14:35:27.087034 4581 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c99fecf0a72469b83a8fdcc35d8c1f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:27 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:46258
I20260811 14:35:27.087064 5035 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c99fecf0a72469b83a8fdcc35d8c1f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:27 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:52266
I20260811 14:35:27.087477 5287 tablet_replica.cc:330] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P cb2428b7f8664aa6ab5db6e748e0d607: stopping tablet replica
I20260811 14:35:27.087567 5287 raft_consensus.cc:2229] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P cb2428b7f8664aa6ab5db6e748e0d607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:27.087621 5287 raft_consensus.cc:2258] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P cb2428b7f8664aa6ab5db6e748e0d607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:27.087913 5287 ts_tablet_manager.cc:1905] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P cb2428b7f8664aa6ab5db6e748e0d607: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:27.088068 4807 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c99fecf0a72469b83a8fdcc35d8c1f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:27 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:33190
I20260811 14:35:27.088325 5293 tablet_replica.cc:330] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52: stopping tablet replica
I20260811 14:35:27.088449 5293 raft_consensus.cc:2229] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:27.088515 5293 raft_consensus.cc:2258] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:27.088798 5293 ts_tablet_manager.cc:1905] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:27.090126 5287 ts_tablet_manager.cc:1918] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P cb2428b7f8664aa6ab5db6e748e0d607: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:35:27.090198 5287 log.cc:1196] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P cb2428b7f8664aa6ab5db6e748e0d607: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-0/wal/wals/3c99fecf0a72469b83a8fdcc35d8c1f3
I20260811 14:35:27.090396 5288 tablet_replica.cc:330] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f: stopping tablet replica
I20260811 14:35:27.090474 5288 raft_consensus.cc:2229] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:35:27.090552 5288 raft_consensus.cc:2258] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:27.090572 5287 ts_tablet_manager.cc:1939] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P cb2428b7f8664aa6ab5db6e748e0d607: Deleting consensus metadata
I20260811 14:35:27.090863 5288 ts_tablet_manager.cc:1905] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:27.090945 4326 catalog_manager.cc:4841] TS cb2428b7f8664aa6ab5db6e748e0d607 (127.2.103.65:35989): tablet 3c99fecf0a72469b83a8fdcc35d8c1f3 (table test-table [id=410e0456a8fb4fbda19f68e7248c0ea6]) successfully deleted
I20260811 14:35:27.091413 5293 ts_tablet_manager.cc:1918] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:35:27.091535 5293 log.cc:1196] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/3c99fecf0a72469b83a8fdcc35d8c1f3
I20260811 14:35:27.091902 5293 ts_tablet_manager.cc:1939] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P 0ca1dae499ef45ab957f06a7c2808f52: Deleting consensus metadata
I20260811 14:35:27.092398 4325 catalog_manager.cc:4841] TS 0ca1dae499ef45ab957f06a7c2808f52 (127.2.103.66:45653): tablet 3c99fecf0a72469b83a8fdcc35d8c1f3 (table test-table [id=410e0456a8fb4fbda19f68e7248c0ea6]) successfully deleted
I20260811 14:35:27.092811 5288 ts_tablet_manager.cc:1918] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:35:27.092907 5288 log.cc:1196] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/3c99fecf0a72469b83a8fdcc35d8c1f3
I20260811 14:35:27.093125 5288 ts_tablet_manager.cc:1939] T 3c99fecf0a72469b83a8fdcc35d8c1f3 P d8078449ee3942d8b30c8d2b0152c43f: Deleting consensus metadata
I20260811 14:35:27.093670 4325 catalog_manager.cc:4841] TS d8078449ee3942d8b30c8d2b0152c43f (127.2.103.67:43551): tablet 3c99fecf0a72469b83a8fdcc35d8c1f3 (table test-table [id=410e0456a8fb4fbda19f68e7248c0ea6]) successfully deleted
Aug 11 14:35:27 dist-test-slave-twwt krb5kdc[4284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458926, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.103.65@KRBTEST.COM
I20260811 14:35:27.102581 5245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:27.090093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46268 (local address 127.2.103.65:35989)
0811 14:35:27.090158 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:27.090161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:27.093612 (+ 3451us) server_negotiation.cc:407] Connection header received
0811 14:35:27.093698 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:27.093700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:27.093718 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:27.093788 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:27.094532 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.095096 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.096179 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.096390 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.096449 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:27.099390 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:27.099405 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:27.099406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:27.099419 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:27.101216 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:27.101639 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:27.101644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:27.101644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:27.101709 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:27.101944 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:27.101944 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:27.101945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:27.102109 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:27.102291 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:27.102340 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 14:35:27.102430 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260811 14:35:27.103039 4681 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46268
I20260811 14:35:27.111028 5235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:27.104233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48810 (local address 127.2.103.126:33913)
0811 14:35:27.104307 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:27.104310 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:27.104340 (+ 30us) server_negotiation.cc:407] Connection header received
0811 14:35:27.104411 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:27.104412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:27.104423 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:27.104484 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:27.104954 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.105773 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.106484 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:27.106869 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:27.106951 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:27.108082 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:27.108083 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:27.108084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:27.108092 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:27.109761 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:27.110114 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:27.110115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:27.110115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:27.110182 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:27.110377 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:27.110378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:27.110378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:27.110521 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:27.110652 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:27.110819 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 14:35:27.110924 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 14:35:27.111248 4347 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48810
I20260811 14:35:27.111783 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 4473
I20260811 14:35:27.127224 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 4704
I20260811 14:35:27.151345 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 4933
I20260811 14:35:27.169395 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 4299
2026-08-11T14:35:27Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2997 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:27 dist-test-slave-twwt krb5kdc[5304](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:27 dist-test-slave-twwt krb5kdc[5304](info): set up 2 sockets
Aug 11 14:35:27 dist-test-slave-twwt krb5kdc[5304](info): commencing operation
krb5kdc: starting...
W20260811 14:35:28.479125 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:35:28 dist-test-slave-twwt krb5kdc[5304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:35:28.614944 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:38795
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:45865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:38795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:28.701102 5325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:28.701325 5325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:28.701349 5325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:28.702562 5325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:28.702630 5325 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 14:35:28.702646 5325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:28.702659 5325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:28.702672 5325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:28.702684 5325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:28.704303 5325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:45865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:38795
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:38795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:28.704589 5325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:28.704860 5325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:28.709964 5332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:28.709973 5330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:28.710011 5335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:28.710333 5325 server_base.cc:1034] running on GCE node
I20260811 14:35:28.710685 5325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:28.711072 5325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:28.712245 5325 hybrid_clock.cc:648] HybridClock initialized: now 1786458928712223 us; error 41 us; skew 500 ppm
Aug 11 14:35:28 dist-test-slave-twwt krb5kdc[5304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458928, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:28.718482 5325 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:35:28.719074 5325 webserver.cc:460] Webserver started at http://127.2.103.126:44763/ using document root <none> and password file <none>
I20260811 14:35:28.719329 5325 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:28.719406 5325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:28.719570 5325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:28.720875 5325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "3859fee308d049e5b74f9e85aa2b051b"
format_stamp: "Formatted at 2026-08-11 14:35:28 on dist-test-slave-twwt"
server_key: "f0340eb56eabf5b7e850451b95129351"
server_key_iv: "af137e40089a27f4845d579a1dec9236"
server_key_version: "encryptionkey@0"
I20260811 14:35:28.721259 5325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "3859fee308d049e5b74f9e85aa2b051b"
format_stamp: "Formatted at 2026-08-11 14:35:28 on dist-test-slave-twwt"
server_key: "f0340eb56eabf5b7e850451b95129351"
server_key_iv: "af137e40089a27f4845d579a1dec9236"
server_key_version: "encryptionkey@0"
I20260811 14:35:28.722905 5325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:35:28.723783 5353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:28.724043 5325 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:35:28.724160 5325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "3859fee308d049e5b74f9e85aa2b051b"
format_stamp: "Formatted at 2026-08-11 14:35:28 on dist-test-slave-twwt"
server_key: "f0340eb56eabf5b7e850451b95129351"
server_key_iv: "af137e40089a27f4845d579a1dec9236"
server_key_version: "encryptionkey@0"
I20260811 14:35:28.724242 5325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:28.741353 5325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:28.743918 5325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:28.744102 5325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:28.749184 5325 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:38795
I20260811 14:35:28.749235 5453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:38795 every 8 connection(s)
I20260811 14:35:28.749588 5325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:28.750263 5454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:28.751350 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 5325
I20260811 14:35:28.751430 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:35:28.751610 2461 external_mini_cluster.cc:1428] Setting key da1e249f4481df9dc27a6f31bf38b97b
I20260811 14:35:28.753371 5454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b: Bootstrap starting.
I20260811 14:35:28.754087 5454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:28.754448 5454 log.cc:824] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:28.755246 5454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b: No bootstrap required, opened a new log
I20260811 14:35:28.757810 5454 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3859fee308d049e5b74f9e85aa2b051b" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38795 } }
I20260811 14:35:28.757959 5454 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:28.757982 5454 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3859fee308d049e5b74f9e85aa2b051b, State: Initialized, Role: FOLLOWER
I20260811 14:35:28.758133 5454 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3859fee308d049e5b74f9e85aa2b051b" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38795 } }
I20260811 14:35:28.758221 5454 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:28.758289 5454 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:28.758343 5454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:28.759126 5454 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3859fee308d049e5b74f9e85aa2b051b" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38795 } }
I20260811 14:35:28.759248 5454 leader_election.cc:302] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3859fee308d049e5b74f9e85aa2b051b; no voters:
Aug 11 14:35:28 dist-test-slave-twwt krb5kdc[5304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:28.759421 5454 leader_election.cc:288] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:28.759519 5460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:28.759649 5460 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b [term 1 LEADER]: Becoming Leader. State: Replica: 3859fee308d049e5b74f9e85aa2b051b, State: Running, Role: LEADER
I20260811 14:35:28.759724 5454 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:28.759745 5460 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3859fee308d049e5b74f9e85aa2b051b" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38795 } }
I20260811 14:35:28.760080 5462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3859fee308d049e5b74f9e85aa2b051b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3859fee308d049e5b74f9e85aa2b051b" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38795 } } }
I20260811 14:35:28.760178 5462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:28.760354 5463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3859fee308d049e5b74f9e85aa2b051b. Latest consensus state: current_term: 1 leader_uuid: "3859fee308d049e5b74f9e85aa2b051b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3859fee308d049e5b74f9e85aa2b051b" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38795 } } }
I20260811 14:35:28.760429 5463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:28.760550 5476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:28.761144 5476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:28.762959 5476 catalog_manager.cc:1347] Generated new cluster ID: c4a57bb44da54d5da230ab8f0100edac
I20260811 14:35:28.763013 5476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:35:28.764073 5458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:28.752216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48364 (local address 127.2.103.126:38795)
0811 14:35:28.752430 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:28.752441 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:28.752474 (+ 33us) server_negotiation.cc:407] Connection header received
0811 14:35:28.753022 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:28.753033 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:28.753100 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:28.753261 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:28.753965 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:28.755413 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:28.756211 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:28.756926 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:28.757044 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:28.760000 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:28.760004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:28.760005 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:28.760018 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:28.762193 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:28.762694 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:28.762697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:28.762698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:28.762823 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:28.763070 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:28.763071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:28.763072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:28.763380 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:28.763534 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:28.763710 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 14:35:28.763875 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":99,"threads_started":1}
I20260811 14:35:28.773758 5476 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:28.774338 5476 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:28.782670 5476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b: Generated new TSK 0
I20260811 14:35:28.782892 5476 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
I20260811 14:35:28.823078 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38795
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:45865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:28.911810 5502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:28.912019 5502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:28.912045 5502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:28.913236 5502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:28.913281 5502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:28.913301 5502 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 14:35:28.913314 5502 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 14:35:28.913331 5502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:35:28.914886 5502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:45865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.103.126:38795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:28.915163 5502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:28.915410 5502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:28.920563 5515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:28.920620 5512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:28.920579 5520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:28.920899 5502 server_base.cc:1034] running on GCE node
I20260811 14:35:28.921103 5502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:28.921367 5502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:28.922535 5502 hybrid_clock.cc:648] HybridClock initialized: now 1786458928922518 us; error 27 us; skew 500 ppm
Aug 11 14:35:28 dist-test-slave-twwt krb5kdc[5304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458928, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:28.925210 5502 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:35:28.925598 5502 webserver.cc:460] Webserver started at http://127.2.103.65:42721/ using document root <none> and password file <none>
I20260811 14:35:28.925778 5502 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:28.925817 5502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:28.925915 5502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:28.927018 5502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8"
format_stamp: "Formatted at 2026-08-11 14:35:28 on dist-test-slave-twwt"
server_key: "e86e46fd1ca149ee8f31133d21617b9b"
server_key_iv: "a889137a0acdec7935484a51e5a079cc"
server_key_version: "encryptionkey@0"
I20260811 14:35:28.927347 5502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8"
format_stamp: "Formatted at 2026-08-11 14:35:28 on dist-test-slave-twwt"
server_key: "e86e46fd1ca149ee8f31133d21617b9b"
server_key_iv: "a889137a0acdec7935484a51e5a079cc"
server_key_version: "encryptionkey@0"
I20260811 14:35:28.928771 5502 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:35:28.929557 5530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:28.929718 5502 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:28.929811 5502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8"
format_stamp: "Formatted at 2026-08-11 14:35:28 on dist-test-slave-twwt"
server_key: "e86e46fd1ca149ee8f31133d21617b9b"
server_key_iv: "a889137a0acdec7935484a51e5a079cc"
server_key_version: "encryptionkey@0"
I20260811 14:35:28.929868 5502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:28.941797 5502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:28.944088 5502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:28.944209 5502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:28.944684 5502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:28.944725 5502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:28.944751 5502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:28.944765 5502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:28.951797 5502 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:34085
I20260811 14:35:28.951911 5711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:34085 every 8 connection(s)
I20260811 14:35:28.952237 5502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:35:28 dist-test-slave-twwt krb5kdc[5304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458928, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:28.955938 5458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:28.945279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:60427 (local address 127.2.103.126:38795)
0811 14:35:28.945340 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:28.945343 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:28.945557 (+ 214us) server_negotiation.cc:407] Connection header received
0811 14:35:28.946193 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:28.946194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:28.946210 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:28.946288 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:28.947311 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:28.948174 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:28.949772 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:28.950145 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:28.950233 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:28.953237 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:28.953238 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:28.953239 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:28.953246 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:28.954517 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:28.955047 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:28.955049 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:28.955049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:28.955109 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:28.955356 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:28.955357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:28.955357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:28.955468 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:28.955521 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:28.955766 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 14:35:28.955874 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 14:35:28.956053 5542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:28.945254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:38795 (local address 127.2.103.65:60427)
0811 14:35:28.945432 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:28.945468 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:28.946019 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:28.946387 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:28.946392 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:28.946851 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:28.947181 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:28.947189 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:28.948281 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:28.948282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:28.949670 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:28.949673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:28.950275 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:28.950276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:28.950432 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:28.951158 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:28.951178 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:28.953115 (+ 1937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:28.954681 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:28.954689 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:28.954691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:28.954938 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:28.955182 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:28.955182 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:28.955183 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:28.955284 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:28.955524 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:28.955529 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:28.955643 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 14:35:28.955721 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 14:35:28.955850 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
I20260811 14:35:28.956627 5712 heartbeater.cc:347] Connected to a master server at 127.2.103.126:38795
I20260811 14:35:28.956732 5712 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:28.956918 5712 heartbeater.cc:511] Master 127.2.103.126:38795 requested a full tablet report, sending...
I20260811 14:35:28.957785 5379 ts_manager.cc:194] Registered new tserver with Master: fe487f13ca3b45bc8ad4cd08c1e604a8 (127.2.103.65:34085)
I20260811 14:35:28.958688 5379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:60427
I20260811 14:35:28.959674 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 5502
I20260811 14:35:28.959780 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:35:28.960041 2461 external_mini_cluster.cc:1428] Setting key c2446cd7368b63c4a51b39170b4b51b1
I20260811 14:35:28.963927 5458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:28.957727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:60175 (local address 127.2.103.126:38795)
0811 14:35:28.957771 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:28.957773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:28.957850 (+ 77us) server_negotiation.cc:407] Connection header received
0811 14:35:28.957957 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:28.957958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:28.957968 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:28.958015 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:28.958371 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:28.959029 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:28.959914 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:28.960196 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:28.960272 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:28.961290 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:28.961291 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:28.961292 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:28.961298 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:28.962713 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:28.963084 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:28.963085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:28.963085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:28.963146 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:28.963368 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:28.963369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:28.963369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:28.963489 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:28.963568 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:28.963744 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 14:35:28.963828 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:35:28.963938 5723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:28.957625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:38795 (local address 127.2.103.65:60175)
0811 14:35:28.957787 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:28.957799 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:28.957865 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:28.958021 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:28.958022 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:28.958216 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:28.958284 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:28.958286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:28.959115 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:28.959116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:28.959790 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:28.959792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:28.960270 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:28.960271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:28.960350 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:28.960867 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:28.960873 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:28.961189 (+ 316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:28.962800 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:28.962801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:28.962802 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:28.962990 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:28.963226 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:28.963227 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:28.963227 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:28.963294 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:28.963559 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:28.963560 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:28.963623 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 14:35:28.963703 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 14:35:28.963835 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for kudu/127.2.103.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
I20260811 14:35:28.993774 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38795
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:45865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:29.080194 5728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:29.080421 5728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:29.080447 5728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:29.081548 5728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:29.081588 5728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:29.081609 5728 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 14:35:29.081620 5728 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 14:35:29.081637 5728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:35:29.083182 5728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:45865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.103.126:38795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:29.083420 5728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:29.083613 5728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:29.088913 5739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:29.088995 5742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:29.089035 5738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:29.089326 5728 server_base.cc:1034] running on GCE node
I20260811 14:35:29.089567 5728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:29.089835 5728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:29.091014 5728 hybrid_clock.cc:648] HybridClock initialized: now 1786458929090987 us; error 39 us; skew 500 ppm
Aug 11 14:35:29 dist-test-slave-twwt krb5kdc[5304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458929, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:29.093775 5728 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:35:29.094331 5728 webserver.cc:460] Webserver started at http://127.2.103.66:42685/ using document root <none> and password file <none>
I20260811 14:35:29.094583 5728 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:29.094693 5728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:29.094856 5728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:29.095990 5728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "4dff8721755c4283803b31c2e3a0ce2f"
format_stamp: "Formatted at 2026-08-11 14:35:29 on dist-test-slave-twwt"
server_key: "e890f3ac712e61dae2f6fa534f131e25"
server_key_iv: "fcce05f756362b1cc6951d92f45c03e4"
server_key_version: "encryptionkey@0"
I20260811 14:35:29.096402 5728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "4dff8721755c4283803b31c2e3a0ce2f"
format_stamp: "Formatted at 2026-08-11 14:35:29 on dist-test-slave-twwt"
server_key: "e890f3ac712e61dae2f6fa534f131e25"
server_key_iv: "fcce05f756362b1cc6951d92f45c03e4"
server_key_version: "encryptionkey@0"
I20260811 14:35:29.097851 5728 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:29.098690 5751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:29.098915 5728 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:29.099025 5728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "4dff8721755c4283803b31c2e3a0ce2f"
format_stamp: "Formatted at 2026-08-11 14:35:29 on dist-test-slave-twwt"
server_key: "e890f3ac712e61dae2f6fa534f131e25"
server_key_iv: "fcce05f756362b1cc6951d92f45c03e4"
server_key_version: "encryptionkey@0"
I20260811 14:35:29.099117 5728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:29.118779 5728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:29.121289 5728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:29.121419 5728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:29.122102 5728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:29.122149 5728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:29.122174 5728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:29.122186 5728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:29.129777 5728 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:39447
I20260811 14:35:29.129860 5943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:39447 every 8 connection(s)
I20260811 14:35:29.130247 5728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
Aug 11 14:35:29 dist-test-slave-twwt krb5kdc[5304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458929, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:29.134310 5458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.122598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:57677 (local address 127.2.103.126:38795)
0811 14:35:29.122698 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:29.122701 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:29.122999 (+ 298us) server_negotiation.cc:407] Connection header received
0811 14:35:29.123601 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:29.123602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:29.123619 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:29.123709 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:29.124866 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.125723 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.127553 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.127892 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.127985 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.131026 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:29.131028 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:29.131028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:29.131036 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:29.132403 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:29.133004 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:29.133005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:29.133006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:29.133069 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:29.133414 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:29.133416 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:29.133417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:29.133570 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:29.133639 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:29.134130 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 14:35:29.134241 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 14:35:29.134464 5765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.122579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:38795 (local address 127.2.103.66:57677)
0811 14:35:29.122835 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:29.122882 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:29.123446 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:29.123808 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:29.123812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:29.124386 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:29.124742 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.124751 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.125848 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.125849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.127448 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.127453 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.128032 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.128033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.128174 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.128872 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:29.128953 (+ 81us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:29.130921 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:29.132501 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:29.132508 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:29.132511 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:29.132895 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:29.133157 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:29.133158 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:29.133158 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:29.133289 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:29.133672 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:29.133680 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:29.133946 (+ 266us) client_negotiation.cc:769] Sending connection context
0811 14:35:29.134050 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 14:35:29.134210 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":151,"threads_started":1}
I20260811 14:35:29.135223 5945 heartbeater.cc:347] Connected to a master server at 127.2.103.126:38795
I20260811 14:35:29.135324 5945 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:29.135547 5945 heartbeater.cc:511] Master 127.2.103.126:38795 requested a full tablet report, sending...
I20260811 14:35:29.136011 5379 ts_manager.cc:194] Registered new tserver with Master: 4dff8721755c4283803b31c2e3a0ce2f (127.2.103.66:39447)
I20260811 14:35:29.136646 5379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:57677
I20260811 14:35:29.140548 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 5728
I20260811 14:35:29.140664 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:35:29.140909 2461 external_mini_cluster.cc:1428] Setting key c2bad9865b044bf0c8dcd0796539340f
I20260811 14:35:29.143177 5458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.136748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:39525 (local address 127.2.103.126:38795)
0811 14:35:29.136800 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:29.136802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:29.136962 (+ 160us) server_negotiation.cc:407] Connection header received
0811 14:35:29.137042 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:29.137043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:29.137054 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:29.137127 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:29.137487 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.137969 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.138716 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.138915 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.138995 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.140087 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:29.140089 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:29.140089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:29.140096 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:29.141787 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:29.142205 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:29.142206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:29.142206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:29.142268 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:29.142502 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:29.142503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:29.142503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:29.142735 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:29.142815 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:29.143006 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 14:35:29.143080 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 14:35:29.143190 5954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.136635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:38795 (local address 127.2.103.66:39525)
0811 14:35:29.136879 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:29.136890 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:29.136979 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:29.137133 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:29.137134 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:29.137342 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:29.137405 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.137406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.138073 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.138074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.138590 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.138591 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.139007 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.139007 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.139090 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.139609 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:29.139620 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:29.139972 (+ 352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:29.141935 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:29.141936 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:29.141937 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:29.142125 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:29.142337 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:29.142338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:29.142338 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:29.142425 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:29.142806 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:29.142807 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:29.142874 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 14:35:29.142959 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 14:35:29.143083 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":173,"threads_started":1}
WARNING: no policy specified for kudu/127.2.103.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
I20260811 14:35:29.177733 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38795
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:45865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:29.268590 5958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:29.268812 5958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:29.268841 5958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:29.269968 5958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:29.270006 5958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:29.270027 5958 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 14:35:29.270040 5958 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 14:35:29.270057 5958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:35:29.271610 5958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:45865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.103.126:38795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:29.271821 5958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:29.272022 5958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:29.277359 5970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:29.277339 5965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:29.277556 5958 server_base.cc:1034] running on GCE node
W20260811 14:35:29.277323 5966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:29.277822 5958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:29.278057 5958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:29.279227 5958 hybrid_clock.cc:648] HybridClock initialized: now 1786458929279196 us; error 46 us; skew 500 ppm
Aug 11 14:35:29 dist-test-slave-twwt krb5kdc[5304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458929, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:29.281914 5958 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:35:29.282342 5958 webserver.cc:460] Webserver started at http://127.2.103.67:38193/ using document root <none> and password file <none>
I20260811 14:35:29.282697 5958 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:29.282783 5958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:29.282930 5958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:29.284123 5958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "b808b4cb2ac343eebfecf1ff12b596f4"
format_stamp: "Formatted at 2026-08-11 14:35:29 on dist-test-slave-twwt"
server_key: "585f965b5ebb698d3e88a8fa9e892a90"
server_key_iv: "6acec6cc7fdc562a481e37ad77b9f8e9"
server_key_version: "encryptionkey@0"
I20260811 14:35:29.284520 5958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "b808b4cb2ac343eebfecf1ff12b596f4"
format_stamp: "Formatted at 2026-08-11 14:35:29 on dist-test-slave-twwt"
server_key: "585f965b5ebb698d3e88a8fa9e892a90"
server_key_iv: "6acec6cc7fdc562a481e37ad77b9f8e9"
server_key_version: "encryptionkey@0"
I20260811 14:35:29.286016 5958 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:35:29.286821 5982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:29.286993 5958 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:35:29.287103 5958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "b808b4cb2ac343eebfecf1ff12b596f4"
format_stamp: "Formatted at 2026-08-11 14:35:29 on dist-test-slave-twwt"
server_key: "585f965b5ebb698d3e88a8fa9e892a90"
server_key_iv: "6acec6cc7fdc562a481e37ad77b9f8e9"
server_key_version: "encryptionkey@0"
I20260811 14:35:29.287180 5958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:29.301569 5958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:29.304311 5958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:29.304489 5958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:29.305040 5958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:29.305104 5958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:29.305142 5958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:29.305174 5958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 14:35:29 dist-test-slave-twwt krb5kdc[5304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458929, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:29.313673 5958 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:46251
I20260811 14:35:29.314059 5958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:35:29.313745 6159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:46251 every 8 connection(s)
I20260811 14:35:29.314879 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 5958
I20260811 14:35:29.314990 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:35:29.315202 2461 external_mini_cluster.cc:1428] Setting key 7275bc71749143a714a282d0b4a300ba
I20260811 14:35:29.317174 5458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.305743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:51967 (local address 127.2.103.126:38795)
0811 14:35:29.305825 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:29.305829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:29.306057 (+ 228us) server_negotiation.cc:407] Connection header received
0811 14:35:29.306828 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:29.306830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:29.306854 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:29.306933 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:29.307978 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.308823 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.310715 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.311067 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.311153 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.314142 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:29.314144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:29.314144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:29.314153 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:29.316004 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:29.316445 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:29.316446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:29.316446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:29.316500 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:29.316684 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:29.316685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:29.316685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:29.316790 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:29.316841 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:29.317030 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 14:35:29.317093 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 14:35:29.317399 5994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.305653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:38795 (local address 127.2.103.67:51967)
0811 14:35:29.305905 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:29.305951 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:29.306591 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:29.307046 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:29.307051 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:29.307494 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:29.307854 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.307864 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.309021 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.309022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.310562 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.310565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.311174 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.311176 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.311352 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.312063 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:29.312073 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:29.314028 (+ 1955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:29.316106 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:29.316114 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:29.316116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:29.316376 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:29.316544 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:29.316544 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:29.316545 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:29.316631 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:29.316830 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:29.316833 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:29.316941 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 14:35:29.317018 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 14:35:29.317169 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":145,"threads_started":1}
I20260811 14:35:29.317931 6160 heartbeater.cc:347] Connected to a master server at 127.2.103.126:38795
I20260811 14:35:29.318049 6160 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:29.318238 6160 heartbeater.cc:511] Master 127.2.103.126:38795 requested a full tablet report, sending...
I20260811 14:35:29.318707 5381 ts_manager.cc:194] Registered new tserver with Master: b808b4cb2ac343eebfecf1ff12b596f4 (127.2.103.67:46251)
I20260811 14:35:29.319104 5381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:51967
I20260811 14:35:29.324723 6175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.319166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:38795 (local address 127.2.103.67:59075)
0811 14:35:29.319330 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:29.319350 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:29.319417 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:29.319588 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:29.319589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:29.319787 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:29.319889 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.319892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.320529 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.320530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.321125 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.321126 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.321483 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.321484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.321543 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.321934 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:29.321939 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:29.322249 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:29.323883 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:29.323884 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:29.323884 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:29.324048 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:29.324228 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:29.324229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:29.324229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:29.324282 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:29.324467 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:29.324467 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:29.324515 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 14:35:29.324566 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 14:35:29.324648 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
I20260811 14:35:29.324731 5458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.319295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:59075 (local address 127.2.103.126:38795)
0811 14:35:29.319356 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:29.319358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:29.319400 (+ 42us) server_negotiation.cc:407] Connection header received
0811 14:35:29.319475 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:29.319476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:29.319486 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:29.319532 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:29.319973 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.320437 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.321217 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.321410 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.321468 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.322337 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:29.322339 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:29.322339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:29.322345 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:29.323811 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:29.324116 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:29.324117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:29.324118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:29.324169 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:29.324324 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:29.324324 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:29.324325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:29.324415 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:29.324462 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:29.324610 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 14:35:29.324670 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 14:35:29.327095 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 14:35:29 dist-test-slave-twwt krb5kdc[5304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 14:35:29 dist-test-slave-twwt krb5kdc[5304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:29.355618 5458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.347933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48372 (local address 127.2.103.126:38795)
0811 14:35:29.348002 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:29.348004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:29.348017 (+ 13us) server_negotiation.cc:407] Connection header received
0811 14:35:29.348104 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:29.348105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:29.348116 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:29.348172 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:29.348836 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.349351 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.350125 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.350334 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.350400 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.352880 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:29.352881 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:29.352882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:29.352889 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:29.354368 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:29.354830 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:29.354831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:29.354831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:29.354889 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:29.355125 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:29.355126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:29.355126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:29.355222 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:29.355312 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:29.355453 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 14:35:29.355517 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 14:35:29.357555 5381 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48372:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:35:29.359612 5381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:35:29.367369 6197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.362383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:46251 (local address 127.0.0.1:43666)
0811 14:35:29.362488 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:29.362533 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:29.362814 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:29.363364 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:29.363369 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:29.363371 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:29.363469 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.363471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.364619 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.364621 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.366112 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.366116 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.367056 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.367057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.367124 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.367148 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 14:35:29.367185 (+ 37us) client_negotiation.cc:240] Negotiation successful
0811 14:35:29.367271 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":40,"thread_start_us":56,"threads_started":1}
I20260811 14:35:29.367730 6201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.362811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42782 (local address 127.2.103.65:34085)
0811 14:35:29.363104 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:29.363108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:29.363474 (+ 366us) server_negotiation.cc:407] Connection header received
0811 14:35:29.363526 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:29.363529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:29.363548 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:29.363623 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:29.363877 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.364929 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.366093 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.366908 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.366995 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.367067 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:29.367272 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 14:35:29.367313 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":101,"threads_started":1}
I20260811 14:35:29.367727 6199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.362414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:39447 (local address 127.0.0.1:46412)
0811 14:35:29.363020 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:29.363045 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:29.363266 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:29.363618 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:29.363619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:29.363622 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:29.363688 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.363689 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.364995 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.364996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.366351 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.366354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.367446 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.367447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.367521 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.367527 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 14:35:29.367583 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 14:35:29.367626 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"mutex_wait_us":145,"thread_start_us":115,"threads_started":1}
I20260811 14:35:29.367384 6198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.362414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:34085 (local address 127.0.0.1:42782)
0811 14:35:29.363352 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:29.363362 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:29.363411 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:29.363699 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:29.363700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:29.363717 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:29.363776 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.363778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.365020 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.365021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.365982 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.365985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.367012 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.367013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.367116 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.367136 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 14:35:29.367232 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 14:35:29.367284 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":917,"mutex_wait_us":3,"thread_start_us":110,"threads_started":1}
I20260811 14:35:29.367918 6200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.362555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43666 (local address 127.2.103.67:46251)
0811 14:35:29.362826 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:29.362833 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:29.362848 (+ 15us) server_negotiation.cc:407] Connection header received
0811 14:35:29.362956 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:29.362959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:29.362978 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:29.363563 (+ 585us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:29.363638 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.364510 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.366215 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.366960 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.367620 (+ 660us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.367694 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:29.367722 (+ 28us) server_negotiation.cc:299] Negotiation successful
0811 14:35:29.367772 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":171,"threads_started":1}
I20260811 14:35:29.368084 6202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.362881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46412 (local address 127.2.103.66:39447)
0811 14:35:29.363044 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:29.363049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:29.363272 (+ 223us) server_negotiation.cc:407] Connection header received
0811 14:35:29.363355 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:29.363358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:29.363376 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:29.363624 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:29.363747 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.364891 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.366457 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.367262 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.367342 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.367405 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:29.367968 (+ 563us) server_negotiation.cc:299] Negotiation successful
0811 14:35:29.368006 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260811 14:35:29.368611 6067 tablet_service.cc:1467] Processing CreateTablet for tablet 8dbca0f972834109a8fa37a4c2cc4d67 (DEFAULT_TABLE table=test-table [id=e8fcaae2bf694480b432c42d0f704016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:29.368731 5836 tablet_service.cc:1467] Processing CreateTablet for tablet 8dbca0f972834109a8fa37a4c2cc4d67 (DEFAULT_TABLE table=test-table [id=e8fcaae2bf694480b432c42d0f704016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:29.369045 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dbca0f972834109a8fa37a4c2cc4d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:29.369045 6067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dbca0f972834109a8fa37a4c2cc4d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:29.368927 5604 tablet_service.cc:1467] Processing CreateTablet for tablet 8dbca0f972834109a8fa37a4c2cc4d67 (DEFAULT_TABLE table=test-table [id=e8fcaae2bf694480b432c42d0f704016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:29.369323 5604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dbca0f972834109a8fa37a4c2cc4d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:29.372757 6203 tablet_bootstrap.cc:492] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4: Bootstrap starting.
I20260811 14:35:29.373116 6205 tablet_bootstrap.cc:492] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f: Bootstrap starting.
I20260811 14:35:29.373708 6205 tablet_bootstrap.cc:654] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:29.374022 6203 tablet_bootstrap.cc:654] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:29.374094 6205 log.cc:824] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:29.374402 6203 log.cc:824] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:29.374872 6204 tablet_bootstrap.cc:492] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8: Bootstrap starting.
I20260811 14:35:29.374933 6205 tablet_bootstrap.cc:492] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f: No bootstrap required, opened a new log
I20260811 14:35:29.375023 6205 ts_tablet_manager.cc:1397] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:35:29.375335 6203 tablet_bootstrap.cc:492] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4: No bootstrap required, opened a new log
I20260811 14:35:29.375415 6203 ts_tablet_manager.cc:1397] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260811 14:35:29.375600 6204 tablet_bootstrap.cc:654] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:29.375967 6204 log.cc:824] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:29.376892 6204 tablet_bootstrap.cc:492] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8: No bootstrap required, opened a new log
I20260811 14:35:29.376977 6204 ts_tablet_manager.cc:1397] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:35:29.376927 6205 raft_consensus.cc:348] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } } peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } }
I20260811 14:35:29.377239 6205 raft_consensus.cc:374] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:29.377290 6205 raft_consensus.cc:729] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dff8721755c4283803b31c2e3a0ce2f, State: Initialized, Role: FOLLOWER
I20260811 14:35:29.377446 6205 consensus_queue.cc:260] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } } peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } }
I20260811 14:35:29.377777 6205 ts_tablet_manager.cc:1428] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:35:29.377803 5945 heartbeater.cc:503] Master 127.2.103.126:38795 was elected leader, sending a full tablet report...
I20260811 14:35:29.377848 6203 raft_consensus.cc:348] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } } peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } }
I20260811 14:35:29.378027 6203 raft_consensus.cc:374] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:29.378067 6203 raft_consensus.cc:729] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b808b4cb2ac343eebfecf1ff12b596f4, State: Initialized, Role: FOLLOWER
I20260811 14:35:29.378229 6203 consensus_queue.cc:260] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } } peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } }
I20260811 14:35:29.378655 6203 ts_tablet_manager.cc:1428] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 14:35:29.378837 6160 heartbeater.cc:503] Master 127.2.103.126:38795 was elected leader, sending a full tablet report...
I20260811 14:35:29.379791 6204 raft_consensus.cc:348] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } } peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } }
I20260811 14:35:29.379930 6204 raft_consensus.cc:374] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:29.379952 6204 raft_consensus.cc:729] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe487f13ca3b45bc8ad4cd08c1e604a8, State: Initialized, Role: FOLLOWER
I20260811 14:35:29.380110 6204 consensus_queue.cc:260] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } } peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } }
I20260811 14:35:29.380404 6204 ts_tablet_manager.cc:1428] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:35:29.380484 5712 heartbeater.cc:503] Master 127.2.103.126:38795 was elected leader, sending a full tablet report...
W20260811 14:35:29.380898 5946 tablet.cc:2369] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:35:29.390533 6212 raft_consensus.cc:482] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:35:29.390760 6212 raft_consensus.cc:504] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } } peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } }
I20260811 14:35:29.391192 6212 leader_election.cc:288] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b808b4cb2ac343eebfecf1ff12b596f4 (127.2.103.67:46251), 4dff8721755c4283803b31c2e3a0ce2f (127.2.103.66:39447)
I20260811 14:35:29.395807 6200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.391451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:48711 (local address 127.2.103.67:46251)
0811 14:35:29.391514 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:29.391517 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:29.391528 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:35:29.391589 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:29.391590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:29.391601 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:29.391661 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:29.391957 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.392991 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.394596 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.395360 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.395431 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.395440 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:29.395682 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 14:35:29.395727 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 14:35:29.395807 6202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.391536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:52905 (local address 127.2.103.66:39447)
0811 14:35:29.391598 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:29.391600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:29.391935 (+ 335us) server_negotiation.cc:407] Connection header received
0811 14:35:29.391968 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:29.391969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:29.391979 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:29.392050 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:29.392232 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.393114 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.394596 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.395325 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.395423 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.395434 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:29.395682 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 14:35:29.395728 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 14:35:29.395839 5542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.391379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:46251 (local address 127.2.103.65:48711)
0811 14:35:29.391469 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:29.391481 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:29.391529 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:29.391669 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:29.391670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:29.391672 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:29.391826 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.391828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.393107 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.393108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.394479 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.394482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.395467 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.395468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.395579 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.395586 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 14:35:29.395660 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 14:35:29.395778 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":34}
I20260811 14:35:29.395843 6213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.391379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:39447 (local address 127.2.103.65:52905)
0811 14:35:29.391647 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:29.391656 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:29.391863 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:29.392091 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:29.392092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:29.392094 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:29.392164 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.392166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.393288 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.393288 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.394479 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.394482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.395467 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.395468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.395579 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.395586 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 14:35:29.395660 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 14:35:29.395729 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":48,"thread_start_us":141,"threads_started":1}
I20260811 14:35:29.396308 5865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dbca0f972834109a8fa37a4c2cc4d67" candidate_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dff8721755c4283803b31c2e3a0ce2f" is_pre_election: true
I20260811 14:35:29.396473 5865 raft_consensus.cc:2454] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe487f13ca3b45bc8ad4cd08c1e604a8 in term 0.
I20260811 14:35:29.396826 6099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dbca0f972834109a8fa37a4c2cc4d67" candidate_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" is_pre_election: true
I20260811 14:35:29.396905 5532 leader_election.cc:302] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dff8721755c4283803b31c2e3a0ce2f, fe487f13ca3b45bc8ad4cd08c1e604a8; no voters:
I20260811 14:35:29.396997 6099 raft_consensus.cc:2454] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe487f13ca3b45bc8ad4cd08c1e604a8 in term 0.
I20260811 14:35:29.397051 6212 raft_consensus.cc:2781] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:35:29.397115 6212 raft_consensus.cc:482] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:35:29.397147 6212 raft_consensus.cc:3037] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:29.397964 6212 raft_consensus.cc:504] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } } peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } }
I20260811 14:35:29.398175 6212 leader_election.cc:288] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [CANDIDATE]: Term 1 election: Requested vote from peers b808b4cb2ac343eebfecf1ff12b596f4 (127.2.103.67:46251), 4dff8721755c4283803b31c2e3a0ce2f (127.2.103.66:39447)
I20260811 14:35:29.398347 5865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dbca0f972834109a8fa37a4c2cc4d67" candidate_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dff8721755c4283803b31c2e3a0ce2f"
I20260811 14:35:29.398453 5865 raft_consensus.cc:3037] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:29.398437 6099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dbca0f972834109a8fa37a4c2cc4d67" candidate_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b808b4cb2ac343eebfecf1ff12b596f4"
I20260811 14:35:29.398559 6099 raft_consensus.cc:3037] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:29.399331 5865 raft_consensus.cc:2454] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe487f13ca3b45bc8ad4cd08c1e604a8 in term 1.
I20260811 14:35:29.399348 6099 raft_consensus.cc:2454] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe487f13ca3b45bc8ad4cd08c1e604a8 in term 1.
I20260811 14:35:29.400050 5533 leader_election.cc:302] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b808b4cb2ac343eebfecf1ff12b596f4, fe487f13ca3b45bc8ad4cd08c1e604a8; no voters:
I20260811 14:35:29.400188 6212 raft_consensus.cc:2781] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:29.400359 6212 raft_consensus.cc:686] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 LEADER]: Becoming Leader. State: Replica: fe487f13ca3b45bc8ad4cd08c1e604a8, State: Running, Role: LEADER
I20260811 14:35:29.400466 6212 consensus_queue.cc:237] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } } peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } }
I20260811 14:35:29.401430 5379 catalog_manager.cc:5495] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to fe487f13ca3b45bc8ad4cd08c1e604a8 (127.2.103.65). New cstate: current_term: 1 leader_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } health_report { overall_health: UNKNOWN } } }
Aug 11 14:35:29 dist-test-slave-twwt krb5kdc[5304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458928, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:29.431502 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.423569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:38795 (local address 127.0.0.1:48384)
0811 14:35:29.423792 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:29.423805 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:29.423854 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:29.423990 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:29.423991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:29.424334 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:29.424388 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.424390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.425104 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.425105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.425719 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.425721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.426048 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.426049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.426090 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.426702 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:29.426707 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:29.428480 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:29.430301 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:29.430309 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:29.430312 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:29.430515 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:29.430775 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:29.430775 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:29.430776 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:29.430843 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:29.431074 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:29.431081 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:29.431126 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 14:35:29.431180 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 14:35:29.431282 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":126,"threads_started":1}
I20260811 14:35:29.431641 5458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.423755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48384 (local address 127.2.103.126:38795)
0811 14:35:29.423830 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:29.423832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:29.423874 (+ 42us) server_negotiation.cc:407] Connection header received
0811 14:35:29.423906 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:29.423907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:29.423916 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:29.423961 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:29.424470 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.425009 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.425801 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.425977 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.426028 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.428575 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:29.428577 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:29.428577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:29.428584 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:29.430208 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:29.430584 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:29.430585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:29.430585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:29.430699 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:29.430903 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:29.430903 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:29.430904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:29.431015 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:29.431067 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:29.431229 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 14:35:29.431424 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":104,"server-negotiator.queue_time_us":46}
I20260811 14:35:29.434165 5379 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:48384:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 14:35:29.434392 5379 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:35:29.436177 5836 tablet_service.cc:1467] Processing CreateTablet for tablet da263d9475b04881ab3d334b806536da (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba5916f6c4994f66b5cd7f85f6a1b87c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 14:35:29.436187 6067 tablet_service.cc:1467] Processing CreateTablet for tablet da263d9475b04881ab3d334b806536da (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba5916f6c4994f66b5cd7f85f6a1b87c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 14:35:29.436218 5604 tablet_service.cc:1467] Processing CreateTablet for tablet da263d9475b04881ab3d334b806536da (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba5916f6c4994f66b5cd7f85f6a1b87c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 14:35:29.436372 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da263d9475b04881ab3d334b806536da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:29.436373 6067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da263d9475b04881ab3d334b806536da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:29.436383 5604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da263d9475b04881ab3d334b806536da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:29.437707 6205 tablet_bootstrap.cc:492] T da263d9475b04881ab3d334b806536da P 4dff8721755c4283803b31c2e3a0ce2f: Bootstrap starting.
I20260811 14:35:29.437996 6203 tablet_bootstrap.cc:492] T da263d9475b04881ab3d334b806536da P b808b4cb2ac343eebfecf1ff12b596f4: Bootstrap starting.
I20260811 14:35:29.437994 6204 tablet_bootstrap.cc:492] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8: Bootstrap starting.
I20260811 14:35:29.438486 6205 tablet_bootstrap.cc:654] T da263d9475b04881ab3d334b806536da P 4dff8721755c4283803b31c2e3a0ce2f: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:29.438486 6204 tablet_bootstrap.cc:654] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:29.438540 6203 tablet_bootstrap.cc:654] T da263d9475b04881ab3d334b806536da P b808b4cb2ac343eebfecf1ff12b596f4: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:29.439297 6204 tablet_bootstrap.cc:492] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8: No bootstrap required, opened a new log
I20260811 14:35:29.439368 6204 ts_tablet_manager.cc:1397] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 14:35:29.439396 6203 tablet_bootstrap.cc:492] T da263d9475b04881ab3d334b806536da P b808b4cb2ac343eebfecf1ff12b596f4: No bootstrap required, opened a new log
I20260811 14:35:29.439443 6203 ts_tablet_manager.cc:1397] T da263d9475b04881ab3d334b806536da P b808b4cb2ac343eebfecf1ff12b596f4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:29.439535 6204 raft_consensus.cc:348] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:29.439597 6203 raft_consensus.cc:348] T da263d9475b04881ab3d334b806536da P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:29.439648 6204 raft_consensus.cc:374] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:29.439678 6204 raft_consensus.cc:729] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe487f13ca3b45bc8ad4cd08c1e604a8, State: Initialized, Role: FOLLOWER
I20260811 14:35:29.439661 6203 raft_consensus.cc:374] T da263d9475b04881ab3d334b806536da P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:29.439706 6203 raft_consensus.cc:729] T da263d9475b04881ab3d334b806536da P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b808b4cb2ac343eebfecf1ff12b596f4, State: Initialized, Role: FOLLOWER
I20260811 14:35:29.439735 6204 consensus_queue.cc:260] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:29.439755 6203 consensus_queue.cc:260] T da263d9475b04881ab3d334b806536da P b808b4cb2ac343eebfecf1ff12b596f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:29.439845 6204 ts_tablet_manager.cc:1428] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 14:35:29.439867 6203 ts_tablet_manager.cc:1428] T da263d9475b04881ab3d334b806536da P b808b4cb2ac343eebfecf1ff12b596f4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:29.439930 6212 tablet_replica.cc:439] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } } }
I20260811 14:35:29.440024 6212 tablet_replica.cc:442] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:35:29.440081 6205 tablet_bootstrap.cc:492] T da263d9475b04881ab3d334b806536da P 4dff8721755c4283803b31c2e3a0ce2f: No bootstrap required, opened a new log
I20260811 14:35:29.440124 6205 ts_tablet_manager.cc:1397] T da263d9475b04881ab3d334b806536da P 4dff8721755c4283803b31c2e3a0ce2f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 14:35:29.440200 6211 tablet_replica.cc:439] T da263d9475b04881ab3d334b806536da P b808b4cb2ac343eebfecf1ff12b596f4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } } }
I20260811 14:35:29.440302 6211 tablet_replica.cc:442] T da263d9475b04881ab3d334b806536da P b808b4cb2ac343eebfecf1ff12b596f4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:35:29.440289 6205 raft_consensus.cc:348] T da263d9475b04881ab3d334b806536da P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:29.440375 6205 raft_consensus.cc:374] T da263d9475b04881ab3d334b806536da P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:29.440398 6205 raft_consensus.cc:729] T da263d9475b04881ab3d334b806536da P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dff8721755c4283803b31c2e3a0ce2f, State: Initialized, Role: FOLLOWER
I20260811 14:35:29.440456 6205 consensus_queue.cc:260] T da263d9475b04881ab3d334b806536da P 4dff8721755c4283803b31c2e3a0ce2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:29.440552 6205 ts_tablet_manager.cc:1428] T da263d9475b04881ab3d334b806536da P 4dff8721755c4283803b31c2e3a0ce2f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:29.440658 6209 tablet_replica.cc:439] T da263d9475b04881ab3d334b806536da P 4dff8721755c4283803b31c2e3a0ce2f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } } }
I20260811 14:35:29.440762 6209 tablet_replica.cc:442] T da263d9475b04881ab3d334b806536da P 4dff8721755c4283803b31c2e3a0ce2f: This TxnStatusTablet replica's current role is: FOLLOWER
W20260811 14:35:29.453133 5714 tablet.cc:2369] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:35:29.496209 6212 raft_consensus.cc:482] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:35:29.496312 6212 raft_consensus.cc:504] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:29.496487 6212 leader_election.cc:288] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dff8721755c4283803b31c2e3a0ce2f (127.2.103.66:39447), b808b4cb2ac343eebfecf1ff12b596f4 (127.2.103.67:46251)
I20260811 14:35:29.496774 5865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da263d9475b04881ab3d334b806536da" candidate_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dff8721755c4283803b31c2e3a0ce2f" is_pre_election: true
I20260811 14:35:29.496867 5865 raft_consensus.cc:2454] T da263d9475b04881ab3d334b806536da P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe487f13ca3b45bc8ad4cd08c1e604a8 in term 0.
I20260811 14:35:29.496863 6099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da263d9475b04881ab3d334b806536da" candidate_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" is_pre_election: true
I20260811 14:35:29.497005 6099 raft_consensus.cc:2454] T da263d9475b04881ab3d334b806536da P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe487f13ca3b45bc8ad4cd08c1e604a8 in term 0.
I20260811 14:35:29.497069 5532 leader_election.cc:302] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dff8721755c4283803b31c2e3a0ce2f, fe487f13ca3b45bc8ad4cd08c1e604a8; no voters:
I20260811 14:35:29.497195 6212 raft_consensus.cc:2781] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:35:29.497272 6212 raft_consensus.cc:482] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:35:29.497288 6212 raft_consensus.cc:3037] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:29.497967 6212 raft_consensus.cc:504] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:29.498140 6212 leader_election.cc:288] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8 [CANDIDATE]: Term 1 election: Requested vote from peers 4dff8721755c4283803b31c2e3a0ce2f (127.2.103.66:39447), b808b4cb2ac343eebfecf1ff12b596f4 (127.2.103.67:46251)
I20260811 14:35:29.498320 5865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da263d9475b04881ab3d334b806536da" candidate_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dff8721755c4283803b31c2e3a0ce2f"
I20260811 14:35:29.498359 6099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da263d9475b04881ab3d334b806536da" candidate_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b808b4cb2ac343eebfecf1ff12b596f4"
I20260811 14:35:29.498432 6099 raft_consensus.cc:3037] T da263d9475b04881ab3d334b806536da P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:29.498432 5865 raft_consensus.cc:3037] T da263d9475b04881ab3d334b806536da P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:29.499155 5865 raft_consensus.cc:2454] T da263d9475b04881ab3d334b806536da P 4dff8721755c4283803b31c2e3a0ce2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe487f13ca3b45bc8ad4cd08c1e604a8 in term 1.
I20260811 14:35:29.499158 6099 raft_consensus.cc:2454] T da263d9475b04881ab3d334b806536da P b808b4cb2ac343eebfecf1ff12b596f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe487f13ca3b45bc8ad4cd08c1e604a8 in term 1.
I20260811 14:35:29.499348 5532 leader_election.cc:302] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dff8721755c4283803b31c2e3a0ce2f, fe487f13ca3b45bc8ad4cd08c1e604a8; no voters:
I20260811 14:35:29.499450 6212 raft_consensus.cc:2781] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:29.499511 6212 raft_consensus.cc:686] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 LEADER]: Becoming Leader. State: Replica: fe487f13ca3b45bc8ad4cd08c1e604a8, State: Running, Role: LEADER
I20260811 14:35:29.499579 6212 consensus_queue.cc:237] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:29.499899 6214 tablet_replica.cc:439] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8: TxnStatusTablet state changed. Reason: New leader fe487f13ca3b45bc8ad4cd08c1e604a8. Latest consensus state: current_term: 1 leader_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } } }
I20260811 14:35:29.500001 6214 tablet_replica.cc:442] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8: This TxnStatusTablet replica's current role is: LEADER
I20260811 14:35:29.500209 6233 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 14:35:29.500341 6233 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 14:35:29.500248 5381 catalog_manager.cc:5495] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to fe487f13ca3b45bc8ad4cd08c1e604a8 (127.2.103.65). New cstate: current_term: 1 leader_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260811 14:35:29.516784 6201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.513887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42784 (local address 127.2.103.65:34085)
0811 14:35:29.513955 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:29.513956 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:29.513964 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:35:29.513998 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:29.513999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:29.514034 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:29.514095 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:29.514267 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.514850 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.515985 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.516173 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.516223 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.516449 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:29.516545 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:29.516601 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:29.516682 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 14:35:29.516703 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 14:35:29.516816 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.513614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:34085 (local address 127.0.0.1:42784)
0811 14:35:29.513704 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:29.513719 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:29.513781 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:29.514132 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:29.514133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:29.514135 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:35:29.514192 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.514194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.514936 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.514937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.515888 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.515891 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.516250 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.516251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.516323 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.516343 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:35:29.516611 (+ 268us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:35:29.516620 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 14:35:29.516674 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 14:35:29.516729 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 14:35:29.519028 5865 raft_consensus.cc:1261] T da263d9475b04881ab3d334b806536da P 4dff8721755c4283803b31c2e3a0ce2f [term 1 FOLLOWER]: Refusing update from remote peer fe487f13ca3b45bc8ad4cd08c1e604a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:29.519052 6099 raft_consensus.cc:1261] T da263d9475b04881ab3d334b806536da P b808b4cb2ac343eebfecf1ff12b596f4 [term 1 FOLLOWER]: Refusing update from remote peer fe487f13ca3b45bc8ad4cd08c1e604a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:29.519418 6212 consensus_queue.cc:1035] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:29.519542 6214 consensus_queue.cc:1035] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:29.520884 6211 tablet_replica.cc:439] T da263d9475b04881ab3d334b806536da P b808b4cb2ac343eebfecf1ff12b596f4: TxnStatusTablet state changed. Reason: New leader fe487f13ca3b45bc8ad4cd08c1e604a8. Latest consensus state: current_term: 1 leader_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } } }
I20260811 14:35:29.521009 6211 tablet_replica.cc:442] T da263d9475b04881ab3d334b806536da P b808b4cb2ac343eebfecf1ff12b596f4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:35:29.521328 6209 tablet_replica.cc:439] T da263d9475b04881ab3d334b806536da P 4dff8721755c4283803b31c2e3a0ce2f: TxnStatusTablet state changed. Reason: New leader fe487f13ca3b45bc8ad4cd08c1e604a8. Latest consensus state: current_term: 1 leader_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } } }
I20260811 14:35:29.521471 6209 tablet_replica.cc:442] T da263d9475b04881ab3d334b806536da P 4dff8721755c4283803b31c2e3a0ce2f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:35:29.522850 6212 tablet_replica.cc:439] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } } }
I20260811 14:35:29.522959 6212 tablet_replica.cc:442] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8: This TxnStatusTablet replica's current role is: LEADER
I20260811 14:35:29.523571 6209 tablet_replica.cc:439] T da263d9475b04881ab3d334b806536da P 4dff8721755c4283803b31c2e3a0ce2f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } } }
I20260811 14:35:29.523665 6209 tablet_replica.cc:442] T da263d9475b04881ab3d334b806536da P 4dff8721755c4283803b31c2e3a0ce2f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:35:29.524003 6214 tablet_replica.cc:439] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } } }
I20260811 14:35:29.524122 6214 tablet_replica.cc:442] T da263d9475b04881ab3d334b806536da P fe487f13ca3b45bc8ad4cd08c1e604a8: This TxnStatusTablet replica's current role is: LEADER
I20260811 14:35:29.523725 6238 mvcc.cc:205] Tried to move back new op lower bound from 7317335775306977280 to 7317335775230746624. Current Snapshot: MvccSnapshot[applied={T|T < 7317335775306977280}]
I20260811 14:35:29.524634 6237 mvcc.cc:205] Tried to move back new op lower bound from 7317335775306977280 to 7317335775230746624. Current Snapshot: MvccSnapshot[applied={T|T < 7317335775306977280 or (T in {7317335775306977280})}]
I20260811 14:35:29.525678 6211 tablet_replica.cc:439] T da263d9475b04881ab3d334b806536da P b808b4cb2ac343eebfecf1ff12b596f4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } } }
I20260811 14:35:29.525771 6211 tablet_replica.cc:442] T da263d9475b04881ab3d334b806536da P b808b4cb2ac343eebfecf1ff12b596f4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:35:29.528903 6201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.526082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42798 (local address 127.2.103.65:34085)
0811 14:35:29.526120 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:29.526122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:29.526131 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:35:29.526194 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:29.526195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:29.526203 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:29.526251 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:29.526441 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.527019 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.528157 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.528462 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.528527 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.528674 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:29.528709 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:29.528762 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:29.528813 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 14:35:29.528835 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 14:35:29.534482 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.531576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:34085 (local address 127.2.103.65:59087)
0811 14:35:29.531740 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:29.531751 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:29.531808 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:29.531936 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:29.531937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:29.531938 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:35:29.531994 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.531996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.532921 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.532922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.533607 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.533609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.534063 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.534064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.534115 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.534129 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:35:29.534322 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:35:29.534324 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:35:29.534355 (+ 31us) client_negotiation.cc:240] Negotiation successful
0811 14:35:29.534389 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
I20260811 14:35:29.534487 6201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.531649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:59087 (local address 127.2.103.65:34085)
0811 14:35:29.531689 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:29.531691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:29.531818 (+ 127us) server_negotiation.cc:407] Connection header received
0811 14:35:29.531843 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:29.531844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:29.531854 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:29.531919 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:29.532059 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.532821 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.533693 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.533985 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.534055 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.534221 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:29.534253 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:29.534308 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:29.534372 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 14:35:29.534390 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 14:35:29.536963 5865 raft_consensus.cc:1261] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f [term 1 FOLLOWER]: Refusing update from remote peer fe487f13ca3b45bc8ad4cd08c1e604a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:29.536963 6099 raft_consensus.cc:1261] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4 [term 1 FOLLOWER]: Refusing update from remote peer fe487f13ca3b45bc8ad4cd08c1e604a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:29.537195 6214 consensus_queue.cc:1035] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:29.537410 6252 consensus_queue.cc:1035] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:29.550197 5379 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:29.550338 5379 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:29.551187 5379 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b: Sending DeleteTablet for 3 replicas of tablet 8dbca0f972834109a8fa37a4c2cc4d67
I20260811 14:35:29.551512 6067 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dbca0f972834109a8fa37a4c2cc4d67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:29 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:43666
I20260811 14:35:29.551601 5836 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dbca0f972834109a8fa37a4c2cc4d67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:29 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:46412
I20260811 14:35:29.551568 5604 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dbca0f972834109a8fa37a4c2cc4d67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:29 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:42782
I20260811 14:35:29.551941 6265 tablet_replica.cc:330] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8: stopping tablet replica
I20260811 14:35:29.552078 6265 raft_consensus.cc:2229] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:35:29.552093 5379 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48372:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 14:35:29.552213 6265 raft_consensus.cc:2258] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 14:35:29.552241 5379 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:35:29.552752 6265 ts_tablet_manager.cc:1905] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:29.553470 6067 tablet_service.cc:1467] Processing CreateTablet for tablet a0b6ceb2b4cf4c2ab557db54fd7aeefd (DEFAULT_TABLE table=test-table [id=472d645297ab41588775df7691ae2dea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:29.553612 6067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0b6ceb2b4cf4c2ab557db54fd7aeefd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:29.554819 6265 ts_tablet_manager.cc:1918] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:35:29.554853 5604 tablet_service.cc:1467] Processing CreateTablet for tablet a0b6ceb2b4cf4c2ab557db54fd7aeefd (DEFAULT_TABLE table=test-table [id=472d645297ab41588775df7691ae2dea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:29.555032 5604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0b6ceb2b4cf4c2ab557db54fd7aeefd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:29.555140 5836 tablet_service.cc:1467] Processing CreateTablet for tablet a0b6ceb2b4cf4c2ab557db54fd7aeefd (DEFAULT_TABLE table=test-table [id=472d645297ab41588775df7691ae2dea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:29.555267 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0b6ceb2b4cf4c2ab557db54fd7aeefd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:29.556388 6205 tablet_bootstrap.cc:492] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f: Bootstrap starting.
I20260811 14:35:29.556421 6203 tablet_bootstrap.cc:492] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4: Bootstrap starting.
I20260811 14:35:29.556458 6204 tablet_bootstrap.cc:492] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P fe487f13ca3b45bc8ad4cd08c1e604a8: Bootstrap starting.
I20260811 14:35:29.556843 6205 tablet_bootstrap.cc:654] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:29.556849 6203 tablet_bootstrap.cc:654] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:29.556933 6204 tablet_bootstrap.cc:654] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P fe487f13ca3b45bc8ad4cd08c1e604a8: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:29.557574 6205 tablet_bootstrap.cc:492] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f: No bootstrap required, opened a new log
I20260811 14:35:29.557637 6205 ts_tablet_manager.cc:1397] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 14:35:29.557600 6265 log.cc:1196] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-0/wal/wals/8dbca0f972834109a8fa37a4c2cc4d67
I20260811 14:35:29.557806 6205 raft_consensus.cc:348] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:29.557893 6205 raft_consensus.cc:374] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:29.557924 6205 raft_consensus.cc:729] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dff8721755c4283803b31c2e3a0ce2f, State: Initialized, Role: FOLLOWER
I20260811 14:35:29.557973 6205 consensus_queue.cc:260] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:29.558089 6205 ts_tablet_manager.cc:1428] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 14:35:29.558188 6203 tablet_bootstrap.cc:492] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4: No bootstrap required, opened a new log
I20260811 14:35:29.558233 6203 ts_tablet_manager.cc:1397] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:35:29.558377 6203 raft_consensus.cc:348] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:29.558450 6203 raft_consensus.cc:374] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:29.558470 6203 raft_consensus.cc:729] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b808b4cb2ac343eebfecf1ff12b596f4, State: Initialized, Role: FOLLOWER
I20260811 14:35:29.558509 6203 consensus_queue.cc:260] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:29.558594 6203 ts_tablet_manager.cc:1428] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:29.558833 6265 ts_tablet_manager.cc:1939] T 8dbca0f972834109a8fa37a4c2cc4d67 P fe487f13ca3b45bc8ad4cd08c1e604a8: Deleting consensus metadata
I20260811 14:35:29.559258 6264 tablet_replica.cc:330] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f: stopping tablet replica
I20260811 14:35:29.559336 5355 catalog_manager.cc:4841] TS fe487f13ca3b45bc8ad4cd08c1e604a8 (127.2.103.65:34085): tablet 8dbca0f972834109a8fa37a4c2cc4d67 (table test-table [id=e8fcaae2bf694480b432c42d0f704016]) successfully deleted
I20260811 14:35:29.559439 6264 raft_consensus.cc:2229] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:29.559576 6264 raft_consensus.cc:2258] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:29.559945 6262 tablet_replica.cc:330] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4: stopping tablet replica
I20260811 14:35:29.560086 6262 raft_consensus.cc:2229] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:29.560103 6264 ts_tablet_manager.cc:1905] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:29.560232 6262 raft_consensus.cc:2258] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:29.560652 6262 ts_tablet_manager.cc:1905] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:29.561251 6204 tablet_bootstrap.cc:492] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P fe487f13ca3b45bc8ad4cd08c1e604a8: No bootstrap required, opened a new log
I20260811 14:35:29.561321 6204 ts_tablet_manager.cc:1397] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P fe487f13ca3b45bc8ad4cd08c1e604a8: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260811 14:35:29.561475 6204 raft_consensus.cc:348] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:29.561540 6204 raft_consensus.cc:374] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:29.561559 6204 raft_consensus.cc:729] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe487f13ca3b45bc8ad4cd08c1e604a8, State: Initialized, Role: FOLLOWER
I20260811 14:35:29.561600 6204 consensus_queue.cc:260] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P fe487f13ca3b45bc8ad4cd08c1e604a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:29.561692 6204 ts_tablet_manager.cc:1428] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P fe487f13ca3b45bc8ad4cd08c1e604a8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:29.562043 6264 ts_tablet_manager.cc:1918] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:35:29.562124 6264 log.cc:1196] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/8dbca0f972834109a8fa37a4c2cc4d67
I20260811 14:35:29.562426 6264 ts_tablet_manager.cc:1939] T 8dbca0f972834109a8fa37a4c2cc4d67 P 4dff8721755c4283803b31c2e3a0ce2f: Deleting consensus metadata
I20260811 14:35:29.562964 5356 catalog_manager.cc:4841] TS 4dff8721755c4283803b31c2e3a0ce2f (127.2.103.66:39447): tablet 8dbca0f972834109a8fa37a4c2cc4d67 (table test-table [id=e8fcaae2bf694480b432c42d0f704016]) successfully deleted
I20260811 14:35:29.563149 6262 ts_tablet_manager.cc:1918] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:35:29.563251 6262 log.cc:1196] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/8dbca0f972834109a8fa37a4c2cc4d67
I20260811 14:35:29.563551 6262 ts_tablet_manager.cc:1939] T 8dbca0f972834109a8fa37a4c2cc4d67 P b808b4cb2ac343eebfecf1ff12b596f4: Deleting consensus metadata
I20260811 14:35:29.564050 5357 catalog_manager.cc:4841] TS b808b4cb2ac343eebfecf1ff12b596f4 (127.2.103.67:46251): tablet 8dbca0f972834109a8fa37a4c2cc4d67 (table test-table [id=e8fcaae2bf694480b432c42d0f704016]) successfully deleted
W20260811 14:35:29.564877 6161 tablet.cc:2369] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:35:29.753940 6209 raft_consensus.cc:482] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:35:29.754097 6209 raft_consensus.cc:504] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:29.754484 6209 leader_election.cc:288] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe487f13ca3b45bc8ad4cd08c1e604a8 (127.2.103.65:34085), b808b4cb2ac343eebfecf1ff12b596f4 (127.2.103.67:46251)
I20260811 14:35:29.758126 6277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.754715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:46251 (local address 127.2.103.66:51923)
0811 14:35:29.754875 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:29.754889 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:29.754994 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:29.755190 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:29.755191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:29.755193 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:29.755276 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.755283 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.756281 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.756282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.757221 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.757224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.757832 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.757833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.757910 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.757915 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:35:29.757974 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 14:35:29.758024 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":28,"thread_start_us":83,"threads_started":1}
I20260811 14:35:29.758177 6200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.754795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:51923 (local address 127.2.103.67:46251)
0811 14:35:29.754868 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:29.754870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:29.754961 (+ 91us) server_negotiation.cc:407] Connection header received
0811 14:35:29.755091 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:29.755093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:29.755104 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:29.755170 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:29.755392 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.756194 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.757321 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.757714 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.757844 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.757854 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:29.758023 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 14:35:29.758087 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 14:35:29.758363 6275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.754717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:34085 (local address 127.2.103.66:40119)
0811 14:35:29.754998 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:29.755007 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:29.755076 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:29.755242 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:29.755243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:29.755246 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:29.755336 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.755338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.756253 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.756254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.757218 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.757221 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.758034 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.758035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.758119 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.758123 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:35:29.758194 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 14:35:29.758247 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":52,"thread_start_us":68,"threads_started":1}
I20260811 14:35:29.758356 6201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.754898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:40119 (local address 127.2.103.65:34085)
0811 14:35:29.754946 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:29.754947 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:29.755053 (+ 106us) server_negotiation.cc:407] Connection header received
0811 14:35:29.755148 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:29.755149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:29.755158 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:29.755207 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:29.755453 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.756161 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.757379 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.757892 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.757980 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.757992 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:29.758225 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 14:35:29.758276 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 14:35:29.758594 5629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0b6ceb2b4cf4c2ab557db54fd7aeefd" candidate_uuid: "4dff8721755c4283803b31c2e3a0ce2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" is_pre_election: true
I20260811 14:35:29.758729 6099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0b6ceb2b4cf4c2ab557db54fd7aeefd" candidate_uuid: "4dff8721755c4283803b31c2e3a0ce2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" is_pre_election: true
I20260811 14:35:29.758834 6099 raft_consensus.cc:2454] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dff8721755c4283803b31c2e3a0ce2f in term 0.
I20260811 14:35:29.758824 5629 raft_consensus.cc:2454] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dff8721755c4283803b31c2e3a0ce2f in term 0.
I20260811 14:35:29.759057 5752 leader_election.cc:302] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dff8721755c4283803b31c2e3a0ce2f, fe487f13ca3b45bc8ad4cd08c1e604a8; no voters:
I20260811 14:35:29.759233 6209 raft_consensus.cc:2781] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:35:29.759284 6209 raft_consensus.cc:482] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:35:29.759300 6209 raft_consensus.cc:3037] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:29.760146 6209 raft_consensus.cc:504] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:29.760365 6209 leader_election.cc:288] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [CANDIDATE]: Term 1 election: Requested vote from peers fe487f13ca3b45bc8ad4cd08c1e604a8 (127.2.103.65:34085), b808b4cb2ac343eebfecf1ff12b596f4 (127.2.103.67:46251)
I20260811 14:35:29.760552 5629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0b6ceb2b4cf4c2ab557db54fd7aeefd" candidate_uuid: "4dff8721755c4283803b31c2e3a0ce2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8"
I20260811 14:35:29.760552 6099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0b6ceb2b4cf4c2ab557db54fd7aeefd" candidate_uuid: "4dff8721755c4283803b31c2e3a0ce2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b808b4cb2ac343eebfecf1ff12b596f4"
I20260811 14:35:29.760634 5629 raft_consensus.cc:3037] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:29.760634 6099 raft_consensus.cc:3037] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:29.761494 5629 raft_consensus.cc:2454] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dff8721755c4283803b31c2e3a0ce2f in term 1.
I20260811 14:35:29.761468 6099 raft_consensus.cc:2454] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dff8721755c4283803b31c2e3a0ce2f in term 1.
I20260811 14:35:29.761674 5752 leader_election.cc:302] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dff8721755c4283803b31c2e3a0ce2f, fe487f13ca3b45bc8ad4cd08c1e604a8; no voters:
I20260811 14:35:29.761806 6209 raft_consensus.cc:2781] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:29.761945 6209 raft_consensus.cc:686] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [term 1 LEADER]: Becoming Leader. State: Replica: 4dff8721755c4283803b31c2e3a0ce2f, State: Running, Role: LEADER
I20260811 14:35:29.762050 6209 consensus_queue.cc:237] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:29.762868 5379 catalog_manager.cc:5495] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dff8721755c4283803b31c2e3a0ce2f (127.2.103.66). New cstate: current_term: 1 leader_uuid: "4dff8721755c4283803b31c2e3a0ce2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260811 14:35:29.792534 6202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.789934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46428 (local address 127.2.103.66:39447)
0811 14:35:29.789979 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:29.789981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:29.790044 (+ 63us) server_negotiation.cc:407] Connection header received
0811 14:35:29.790111 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:29.790112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:29.790132 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:29.790198 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:29.790280 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.790834 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.791670 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.791886 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.791973 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.792136 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:29.792256 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:29.792348 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:29.792391 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 14:35:29.792413 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 14:35:29.797395 6201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.795215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:54813 (local address 127.2.103.65:34085)
0811 14:35:29.795259 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:29.795261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:29.795381 (+ 120us) server_negotiation.cc:407] Connection header received
0811 14:35:29.795418 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:29.795419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:29.795428 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:29.795475 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:29.795566 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.796037 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.796803 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.796983 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.797033 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.797192 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:29.797218 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:29.797256 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:29.797319 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 14:35:29.797334 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:35:29.797430 6287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.795129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:34085 (local address 127.2.103.66:54813)
0811 14:35:29.795299 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:29.795309 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:29.795377 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:29.795478 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:29.795478 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:29.795479 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:35:29.795531 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.795532 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.796121 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.796122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.796722 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.796724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.797054 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.797055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.797103 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.797117 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:35:29.797276 (+ 159us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:35:29.797277 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 14:35:29.797308 (+ 31us) client_negotiation.cc:240] Negotiation successful
0811 14:35:29.797356 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":97,"threads_started":1}
I20260811 14:35:29.799870 6099 raft_consensus.cc:1261] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4 [term 1 FOLLOWER]: Refusing update from remote peer 4dff8721755c4283803b31c2e3a0ce2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:29.799959 5629 raft_consensus.cc:1261] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 FOLLOWER]: Refusing update from remote peer 4dff8721755c4283803b31c2e3a0ce2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:29.800194 6209 consensus_queue.cc:1035] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:29.800657 6280 consensus_queue.cc:1035] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:29.802385 6239 mvcc.cc:205] Tried to move back new op lower bound from 7317335776458162176 to 7317335776305860608. Current Snapshot: MvccSnapshot[applied={T|T < 7317335776458162176}]
I20260811 14:35:29.808481 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.805677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:39447 (local address 127.2.103.65:37359)
0811 14:35:29.805763 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:29.805776 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:29.805820 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:29.805950 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:29.805951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:29.805952 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:35:29.805999 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.806001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.806876 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.806877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.807452 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:29.807454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.807968 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.807969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:29.808041 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.808068 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:35:29.808279 (+ 211us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:35:29.808281 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:35:29.808333 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 14:35:29.808398 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 14:35:29.808473 6202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:29.805765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:37359 (local address 127.2.103.66:39447)
0811 14:35:29.805806 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:29.805808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:29.805818 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:35:29.805874 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:29.805875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:29.805886 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:29.805943 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:29.806074 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.806777 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.807577 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:29.807906 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:29.807986 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:29.808168 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:29.808206 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:29.808257 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:29.808368 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 14:35:29.808394 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260811 14:35:29.808842 6238 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 14:35:31.334656 5379 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:31.334760 5379 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:31.335601 5379 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b: Sending DeleteTablet for 3 replicas of tablet a0b6ceb2b4cf4c2ab557db54fd7aeefd
I20260811 14:35:31.335875 5604 tablet_service.cc:1514] Processing DeleteTablet for tablet a0b6ceb2b4cf4c2ab557db54fd7aeefd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:31 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:42782
I20260811 14:35:31.335913 5836 tablet_service.cc:1514] Processing DeleteTablet for tablet a0b6ceb2b4cf4c2ab557db54fd7aeefd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:31 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:46412
I20260811 14:35:31.336124 6326 tablet_replica.cc:330] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f: stopping tablet replica
I20260811 14:35:31.336126 6067 tablet_service.cc:1514] Processing DeleteTablet for tablet a0b6ceb2b4cf4c2ab557db54fd7aeefd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:31 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:43666
I20260811 14:35:31.336297 6326 raft_consensus.cc:2229] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:35:31.336349 6327 tablet_replica.cc:330] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4: stopping tablet replica
I20260811 14:35:31.336364 6325 tablet_replica.cc:330] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P fe487f13ca3b45bc8ad4cd08c1e604a8: stopping tablet replica
I20260811 14:35:31.336414 6325 raft_consensus.cc:2229] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:31.336447 6325 raft_consensus.cc:2258] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:31.336476 6327 raft_consensus.cc:2229] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:31.336490 6326 raft_consensus.cc:2258] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:31.336558 6327 raft_consensus.cc:2258] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:31.336786 6326 ts_tablet_manager.cc:1905] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:31.336824 6327 ts_tablet_manager.cc:1905] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:31.336895 6325 ts_tablet_manager.cc:1905] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P fe487f13ca3b45bc8ad4cd08c1e604a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:31.338893 6326 ts_tablet_manager.cc:1918] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 14:35:31.338904 6327 ts_tablet_manager.cc:1918] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 14:35:31.338979 6327 log.cc:1196] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/a0b6ceb2b4cf4c2ab557db54fd7aeefd
I20260811 14:35:31.339007 6326 log.cc:1196] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/a0b6ceb2b4cf4c2ab557db54fd7aeefd
I20260811 14:35:31.339157 6325 ts_tablet_manager.cc:1918] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P fe487f13ca3b45bc8ad4cd08c1e604a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 14:35:31.339303 6326 ts_tablet_manager.cc:1939] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P 4dff8721755c4283803b31c2e3a0ce2f: Deleting consensus metadata
I20260811 14:35:31.339306 6325 log.cc:1196] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P fe487f13ca3b45bc8ad4cd08c1e604a8: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-0/wal/wals/a0b6ceb2b4cf4c2ab557db54fd7aeefd
I20260811 14:35:31.339308 6327 ts_tablet_manager.cc:1939] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P b808b4cb2ac343eebfecf1ff12b596f4: Deleting consensus metadata
I20260811 14:35:31.339572 6325 ts_tablet_manager.cc:1939] T a0b6ceb2b4cf4c2ab557db54fd7aeefd P fe487f13ca3b45bc8ad4cd08c1e604a8: Deleting consensus metadata
I20260811 14:35:31.339675 5357 catalog_manager.cc:4841] TS b808b4cb2ac343eebfecf1ff12b596f4 (127.2.103.67:46251): tablet a0b6ceb2b4cf4c2ab557db54fd7aeefd (table test-table [id=472d645297ab41588775df7691ae2dea]) successfully deleted
I20260811 14:35:31.339758 5356 catalog_manager.cc:4841] TS 4dff8721755c4283803b31c2e3a0ce2f (127.2.103.66:39447): tablet a0b6ceb2b4cf4c2ab557db54fd7aeefd (table test-table [id=472d645297ab41588775df7691ae2dea]) successfully deleted
I20260811 14:35:31.340019 5355 catalog_manager.cc:4841] TS fe487f13ca3b45bc8ad4cd08c1e604a8 (127.2.103.65:34085): tablet a0b6ceb2b4cf4c2ab557db54fd7aeefd (table test-table [id=472d645297ab41588775df7691ae2dea]) successfully deleted
Aug 11 14:35:31 dist-test-slave-twwt krb5kdc[5304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458931, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 14:35:31 dist-test-slave-twwt krb5kdc[5304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458931, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:31.366962 6337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:31.358683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48392 (local address 127.2.103.126:38795)
0811 14:35:31.358880 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:31.358882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:31.358975 (+ 93us) server_negotiation.cc:407] Connection header received
0811 14:35:31.359102 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:31.359103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:31.359115 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:31.359205 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:31.359852 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:31.360485 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:31.361293 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:31.361537 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:31.361612 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:31.364093 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:31.364095 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:31.364095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:31.364102 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:31.365759 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:31.366144 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:31.366145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:31.366146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:31.366201 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:31.366400 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:31.366401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:31.366401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:31.366508 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:31.366586 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:31.366810 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 14:35:31.366885 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
I20260811 14:35:31.368758 5379 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:35:31.368955 5379 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:35:31.370467 5604 tablet_service.cc:1467] Processing CreateTablet for tablet aeacb0e6cb12494f88ab90165923719e (DEFAULT_TABLE table=test-table [id=6199f26973fd43d098f3e153e49e7b7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:31.370484 6067 tablet_service.cc:1467] Processing CreateTablet for tablet aeacb0e6cb12494f88ab90165923719e (DEFAULT_TABLE table=test-table [id=6199f26973fd43d098f3e153e49e7b7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:31.370528 5836 tablet_service.cc:1467] Processing CreateTablet for tablet aeacb0e6cb12494f88ab90165923719e (DEFAULT_TABLE table=test-table [id=6199f26973fd43d098f3e153e49e7b7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:31.370648 6067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeacb0e6cb12494f88ab90165923719e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:31.370647 5604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeacb0e6cb12494f88ab90165923719e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:31.370723 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeacb0e6cb12494f88ab90165923719e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:31.371771 6342 tablet_bootstrap.cc:492] T aeacb0e6cb12494f88ab90165923719e P 4dff8721755c4283803b31c2e3a0ce2f: Bootstrap starting.
I20260811 14:35:31.372128 6342 tablet_bootstrap.cc:654] T aeacb0e6cb12494f88ab90165923719e P 4dff8721755c4283803b31c2e3a0ce2f: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:31.372211 6345 tablet_bootstrap.cc:492] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8: Bootstrap starting.
I20260811 14:35:31.372223 6344 tablet_bootstrap.cc:492] T aeacb0e6cb12494f88ab90165923719e P b808b4cb2ac343eebfecf1ff12b596f4: Bootstrap starting.
I20260811 14:35:31.372656 6345 tablet_bootstrap.cc:654] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:31.372660 6344 tablet_bootstrap.cc:654] T aeacb0e6cb12494f88ab90165923719e P b808b4cb2ac343eebfecf1ff12b596f4: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:31.372970 6342 tablet_bootstrap.cc:492] T aeacb0e6cb12494f88ab90165923719e P 4dff8721755c4283803b31c2e3a0ce2f: No bootstrap required, opened a new log
I20260811 14:35:31.373039 6342 ts_tablet_manager.cc:1397] T aeacb0e6cb12494f88ab90165923719e P 4dff8721755c4283803b31c2e3a0ce2f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:31.373188 6342 raft_consensus.cc:348] T aeacb0e6cb12494f88ab90165923719e P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:31.373292 6342 raft_consensus.cc:374] T aeacb0e6cb12494f88ab90165923719e P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:31.373342 6342 raft_consensus.cc:729] T aeacb0e6cb12494f88ab90165923719e P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dff8721755c4283803b31c2e3a0ce2f, State: Initialized, Role: FOLLOWER
I20260811 14:35:31.373349 6345 tablet_bootstrap.cc:492] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8: No bootstrap required, opened a new log
I20260811 14:35:31.373423 6345 ts_tablet_manager.cc:1397] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:31.373402 6342 consensus_queue.cc:260] T aeacb0e6cb12494f88ab90165923719e P 4dff8721755c4283803b31c2e3a0ce2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:31.373438 6344 tablet_bootstrap.cc:492] T aeacb0e6cb12494f88ab90165923719e P b808b4cb2ac343eebfecf1ff12b596f4: No bootstrap required, opened a new log
I20260811 14:35:31.373481 6344 ts_tablet_manager.cc:1397] T aeacb0e6cb12494f88ab90165923719e P b808b4cb2ac343eebfecf1ff12b596f4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:31.373515 6342 ts_tablet_manager.cc:1428] T aeacb0e6cb12494f88ab90165923719e P 4dff8721755c4283803b31c2e3a0ce2f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:31.373548 6345 raft_consensus.cc:348] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:31.373618 6345 raft_consensus.cc:374] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:31.373582 6344 raft_consensus.cc:348] T aeacb0e6cb12494f88ab90165923719e P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:31.373639 6345 raft_consensus.cc:729] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe487f13ca3b45bc8ad4cd08c1e604a8, State: Initialized, Role: FOLLOWER
I20260811 14:35:31.373646 6344 raft_consensus.cc:374] T aeacb0e6cb12494f88ab90165923719e P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:31.373667 6344 raft_consensus.cc:729] T aeacb0e6cb12494f88ab90165923719e P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b808b4cb2ac343eebfecf1ff12b596f4, State: Initialized, Role: FOLLOWER
I20260811 14:35:31.373697 6345 consensus_queue.cc:260] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:31.373744 6344 consensus_queue.cc:260] T aeacb0e6cb12494f88ab90165923719e P b808b4cb2ac343eebfecf1ff12b596f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:31.373808 6345 ts_tablet_manager.cc:1428] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:31.373962 6344 ts_tablet_manager.cc:1428] T aeacb0e6cb12494f88ab90165923719e P b808b4cb2ac343eebfecf1ff12b596f4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:31.423127 6257 raft_consensus.cc:482] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:35:31.423372 6257 raft_consensus.cc:504] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:31.423620 6257 leader_election.cc:288] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dff8721755c4283803b31c2e3a0ce2f (127.2.103.66:39447), b808b4cb2ac343eebfecf1ff12b596f4 (127.2.103.67:46251)
I20260811 14:35:31.423902 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeacb0e6cb12494f88ab90165923719e" candidate_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dff8721755c4283803b31c2e3a0ce2f" is_pre_election: true
I20260811 14:35:31.423914 6099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeacb0e6cb12494f88ab90165923719e" candidate_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" is_pre_election: true
I20260811 14:35:31.424024 5875 raft_consensus.cc:2454] T aeacb0e6cb12494f88ab90165923719e P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe487f13ca3b45bc8ad4cd08c1e604a8 in term 0.
I20260811 14:35:31.424067 6099 raft_consensus.cc:2454] T aeacb0e6cb12494f88ab90165923719e P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe487f13ca3b45bc8ad4cd08c1e604a8 in term 0.
I20260811 14:35:31.424258 5532 leader_election.cc:302] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dff8721755c4283803b31c2e3a0ce2f, fe487f13ca3b45bc8ad4cd08c1e604a8; no voters:
I20260811 14:35:31.424401 6257 raft_consensus.cc:2781] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:35:31.424485 6257 raft_consensus.cc:482] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:35:31.424517 6257 raft_consensus.cc:3037] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:31.425344 6257 raft_consensus.cc:504] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:31.425526 6257 leader_election.cc:288] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [CANDIDATE]: Term 1 election: Requested vote from peers 4dff8721755c4283803b31c2e3a0ce2f (127.2.103.66:39447), b808b4cb2ac343eebfecf1ff12b596f4 (127.2.103.67:46251)
I20260811 14:35:31.425717 6099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeacb0e6cb12494f88ab90165923719e" candidate_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b808b4cb2ac343eebfecf1ff12b596f4"
I20260811 14:35:31.425738 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeacb0e6cb12494f88ab90165923719e" candidate_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dff8721755c4283803b31c2e3a0ce2f"
I20260811 14:35:31.425835 6099 raft_consensus.cc:3037] T aeacb0e6cb12494f88ab90165923719e P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:31.425836 5875 raft_consensus.cc:3037] T aeacb0e6cb12494f88ab90165923719e P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:31.426676 6099 raft_consensus.cc:2454] T aeacb0e6cb12494f88ab90165923719e P b808b4cb2ac343eebfecf1ff12b596f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe487f13ca3b45bc8ad4cd08c1e604a8 in term 1.
I20260811 14:35:31.426851 5875 raft_consensus.cc:2454] T aeacb0e6cb12494f88ab90165923719e P 4dff8721755c4283803b31c2e3a0ce2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe487f13ca3b45bc8ad4cd08c1e604a8 in term 1.
I20260811 14:35:31.426887 5533 leader_election.cc:302] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b808b4cb2ac343eebfecf1ff12b596f4, fe487f13ca3b45bc8ad4cd08c1e604a8; no voters:
I20260811 14:35:31.426992 6257 raft_consensus.cc:2781] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:31.427052 6257 raft_consensus.cc:686] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 LEADER]: Becoming Leader. State: Replica: fe487f13ca3b45bc8ad4cd08c1e604a8, State: Running, Role: LEADER
I20260811 14:35:31.427106 6257 consensus_queue.cc:237] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:31.427999 5381 catalog_manager.cc:5495] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to fe487f13ca3b45bc8ad4cd08c1e604a8 (127.2.103.65). New cstate: current_term: 1 leader_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260811 14:35:31.436414 6359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:31.433707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42806 (local address 127.2.103.65:34085)
0811 14:35:31.433904 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:31.433906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:31.433913 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:35:31.433943 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:31.433943 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:31.433954 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:31.434037 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:31.434213 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:31.434782 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:31.435662 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:31.435891 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:31.435969 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:31.436128 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:31.436166 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:31.436226 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:31.436288 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 14:35:31.436314 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":101,"threads_started":1}
I20260811 14:35:31.439146 5875 raft_consensus.cc:1261] T aeacb0e6cb12494f88ab90165923719e P 4dff8721755c4283803b31c2e3a0ce2f [term 1 FOLLOWER]: Refusing update from remote peer fe487f13ca3b45bc8ad4cd08c1e604a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:31.439211 6099 raft_consensus.cc:1261] T aeacb0e6cb12494f88ab90165923719e P b808b4cb2ac343eebfecf1ff12b596f4 [term 1 FOLLOWER]: Refusing update from remote peer fe487f13ca3b45bc8ad4cd08c1e604a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:31.439420 6309 consensus_queue.cc:1035] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:31.439561 6257 consensus_queue.cc:1035] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:31.450265 5381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:31.450408 5381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:31.451184 5381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b: Sending DeleteTablet for 3 replicas of tablet aeacb0e6cb12494f88ab90165923719e
I20260811 14:35:31.451462 5604 tablet_service.cc:1514] Processing DeleteTablet for tablet aeacb0e6cb12494f88ab90165923719e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:31 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:42782
I20260811 14:35:31.451472 5836 tablet_service.cc:1514] Processing DeleteTablet for tablet aeacb0e6cb12494f88ab90165923719e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:31 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:46412
I20260811 14:35:31.451561 6067 tablet_service.cc:1514] Processing DeleteTablet for tablet aeacb0e6cb12494f88ab90165923719e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:31 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:43666
I20260811 14:35:31.451589 6325 tablet_replica.cc:330] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8: stopping tablet replica
I20260811 14:35:31.451656 6327 tablet_replica.cc:330] T aeacb0e6cb12494f88ab90165923719e P b808b4cb2ac343eebfecf1ff12b596f4: stopping tablet replica
I20260811 14:35:31.451656 6325 raft_consensus.cc:2229] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:35:31.451722 6325 raft_consensus.cc:2258] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:31.451705 6327 raft_consensus.cc:2229] T aeacb0e6cb12494f88ab90165923719e P b808b4cb2ac343eebfecf1ff12b596f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:31.451810 6326 tablet_replica.cc:330] T aeacb0e6cb12494f88ab90165923719e P 4dff8721755c4283803b31c2e3a0ce2f: stopping tablet replica
I20260811 14:35:31.451822 6327 raft_consensus.cc:2258] T aeacb0e6cb12494f88ab90165923719e P b808b4cb2ac343eebfecf1ff12b596f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:31.451879 6326 raft_consensus.cc:2229] T aeacb0e6cb12494f88ab90165923719e P 4dff8721755c4283803b31c2e3a0ce2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:31.451922 6326 raft_consensus.cc:2258] T aeacb0e6cb12494f88ab90165923719e P 4dff8721755c4283803b31c2e3a0ce2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:31.451913 5381 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:35:31.452080 5381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:35:31.452310 6325 ts_tablet_manager.cc:1905] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:31.452350 6326 ts_tablet_manager.cc:1905] T aeacb0e6cb12494f88ab90165923719e P 4dff8721755c4283803b31c2e3a0ce2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:31.453373 6067 tablet_service.cc:1467] Processing CreateTablet for tablet 218a00c37c2d42ff8845efd8cb93f2ef (DEFAULT_TABLE table=test-table [id=68b7e58173e141588d90a7c71e99cd4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:31.453498 6067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 218a00c37c2d42ff8845efd8cb93f2ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:31.454349 6326 ts_tablet_manager.cc:1918] T aeacb0e6cb12494f88ab90165923719e P 4dff8721755c4283803b31c2e3a0ce2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:35:31.454362 6325 ts_tablet_manager.cc:1918] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:35:31.454406 6326 log.cc:1196] T aeacb0e6cb12494f88ab90165923719e P 4dff8721755c4283803b31c2e3a0ce2f: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/aeacb0e6cb12494f88ab90165923719e
I20260811 14:35:31.454421 6325 log.cc:1196] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-0/wal/wals/aeacb0e6cb12494f88ab90165923719e
I20260811 14:35:31.454591 6326 ts_tablet_manager.cc:1939] T aeacb0e6cb12494f88ab90165923719e P 4dff8721755c4283803b31c2e3a0ce2f: Deleting consensus metadata
I20260811 14:35:31.454676 6325 ts_tablet_manager.cc:1939] T aeacb0e6cb12494f88ab90165923719e P fe487f13ca3b45bc8ad4cd08c1e604a8: Deleting consensus metadata
I20260811 14:35:31.455075 6344 tablet_bootstrap.cc:492] T 218a00c37c2d42ff8845efd8cb93f2ef P b808b4cb2ac343eebfecf1ff12b596f4: Bootstrap starting.
I20260811 14:35:31.455030 5355 catalog_manager.cc:4841] TS fe487f13ca3b45bc8ad4cd08c1e604a8 (127.2.103.65:34085): tablet aeacb0e6cb12494f88ab90165923719e (table test-table [id=6199f26973fd43d098f3e153e49e7b7e]) successfully deleted
I20260811 14:35:31.455250 5836 tablet_service.cc:1467] Processing CreateTablet for tablet 218a00c37c2d42ff8845efd8cb93f2ef (DEFAULT_TABLE table=test-table [id=68b7e58173e141588d90a7c71e99cd4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:31.455371 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 218a00c37c2d42ff8845efd8cb93f2ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:31.455520 6327 ts_tablet_manager.cc:1905] T aeacb0e6cb12494f88ab90165923719e P b808b4cb2ac343eebfecf1ff12b596f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:31.455569 6344 tablet_bootstrap.cc:654] T 218a00c37c2d42ff8845efd8cb93f2ef P b808b4cb2ac343eebfecf1ff12b596f4: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:31.456370 6344 tablet_bootstrap.cc:492] T 218a00c37c2d42ff8845efd8cb93f2ef P b808b4cb2ac343eebfecf1ff12b596f4: No bootstrap required, opened a new log
I20260811 14:35:31.456420 6344 ts_tablet_manager.cc:1397] T 218a00c37c2d42ff8845efd8cb93f2ef P b808b4cb2ac343eebfecf1ff12b596f4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:31.456570 6344 raft_consensus.cc:348] T 218a00c37c2d42ff8845efd8cb93f2ef P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:31.456662 6344 raft_consensus.cc:374] T 218a00c37c2d42ff8845efd8cb93f2ef P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:31.456686 6342 tablet_bootstrap.cc:492] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f: Bootstrap starting.
I20260811 14:35:31.456689 6344 raft_consensus.cc:729] T 218a00c37c2d42ff8845efd8cb93f2ef P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b808b4cb2ac343eebfecf1ff12b596f4, State: Initialized, Role: FOLLOWER
I20260811 14:35:31.456764 6344 consensus_queue.cc:260] T 218a00c37c2d42ff8845efd8cb93f2ef P b808b4cb2ac343eebfecf1ff12b596f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:31.456869 6344 ts_tablet_manager.cc:1428] T 218a00c37c2d42ff8845efd8cb93f2ef P b808b4cb2ac343eebfecf1ff12b596f4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:31.457006 6327 ts_tablet_manager.cc:1918] T aeacb0e6cb12494f88ab90165923719e P b808b4cb2ac343eebfecf1ff12b596f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:35:31.457059 6327 log.cc:1196] T aeacb0e6cb12494f88ab90165923719e P b808b4cb2ac343eebfecf1ff12b596f4: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/aeacb0e6cb12494f88ab90165923719e
I20260811 14:35:31.457080 6342 tablet_bootstrap.cc:654] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:31.457319 6327 ts_tablet_manager.cc:1939] T aeacb0e6cb12494f88ab90165923719e P b808b4cb2ac343eebfecf1ff12b596f4: Deleting consensus metadata
I20260811 14:35:31.457623 5604 tablet_service.cc:1467] Processing CreateTablet for tablet 218a00c37c2d42ff8845efd8cb93f2ef (DEFAULT_TABLE table=test-table [id=68b7e58173e141588d90a7c71e99cd4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:31.457659 5357 catalog_manager.cc:4841] TS b808b4cb2ac343eebfecf1ff12b596f4 (127.2.103.67:46251): tablet aeacb0e6cb12494f88ab90165923719e (table test-table [id=6199f26973fd43d098f3e153e49e7b7e]) successfully deleted
I20260811 14:35:31.457682 6342 tablet_bootstrap.cc:492] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f: No bootstrap required, opened a new log
I20260811 14:35:31.457731 6342 ts_tablet_manager.cc:1397] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:31.457772 5604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 218a00c37c2d42ff8845efd8cb93f2ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:31.457844 6342 raft_consensus.cc:348] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:31.457897 6342 raft_consensus.cc:374] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:31.457912 6342 raft_consensus.cc:729] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dff8721755c4283803b31c2e3a0ce2f, State: Initialized, Role: FOLLOWER
I20260811 14:35:31.457978 6342 consensus_queue.cc:260] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:31.458077 6342 ts_tablet_manager.cc:1428] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:31.458765 5356 catalog_manager.cc:4841] TS 4dff8721755c4283803b31c2e3a0ce2f (127.2.103.66:39447): tablet aeacb0e6cb12494f88ab90165923719e (table test-table [id=6199f26973fd43d098f3e153e49e7b7e]) successfully deleted
I20260811 14:35:31.459472 6345 tablet_bootstrap.cc:492] T 218a00c37c2d42ff8845efd8cb93f2ef P fe487f13ca3b45bc8ad4cd08c1e604a8: Bootstrap starting.
I20260811 14:35:31.459863 6345 tablet_bootstrap.cc:654] T 218a00c37c2d42ff8845efd8cb93f2ef P fe487f13ca3b45bc8ad4cd08c1e604a8: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:31.460525 6345 tablet_bootstrap.cc:492] T 218a00c37c2d42ff8845efd8cb93f2ef P fe487f13ca3b45bc8ad4cd08c1e604a8: No bootstrap required, opened a new log
I20260811 14:35:31.460567 6345 ts_tablet_manager.cc:1397] T 218a00c37c2d42ff8845efd8cb93f2ef P fe487f13ca3b45bc8ad4cd08c1e604a8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:31.460650 6345 raft_consensus.cc:348] T 218a00c37c2d42ff8845efd8cb93f2ef P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:31.460700 6345 raft_consensus.cc:374] T 218a00c37c2d42ff8845efd8cb93f2ef P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:31.460713 6345 raft_consensus.cc:729] T 218a00c37c2d42ff8845efd8cb93f2ef P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe487f13ca3b45bc8ad4cd08c1e604a8, State: Initialized, Role: FOLLOWER
I20260811 14:35:31.460779 6345 consensus_queue.cc:260] T 218a00c37c2d42ff8845efd8cb93f2ef P fe487f13ca3b45bc8ad4cd08c1e604a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:31.460872 6345 ts_tablet_manager.cc:1428] T 218a00c37c2d42ff8845efd8cb93f2ef P fe487f13ca3b45bc8ad4cd08c1e604a8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:31.514531 6209 raft_consensus.cc:482] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:35:31.514694 6209 raft_consensus.cc:504] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:31.514891 6209 leader_election.cc:288] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe487f13ca3b45bc8ad4cd08c1e604a8 (127.2.103.65:34085), b808b4cb2ac343eebfecf1ff12b596f4 (127.2.103.67:46251)
I20260811 14:35:31.515136 5629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "218a00c37c2d42ff8845efd8cb93f2ef" candidate_uuid: "4dff8721755c4283803b31c2e3a0ce2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" is_pre_election: true
I20260811 14:35:31.515226 5629 raft_consensus.cc:2454] T 218a00c37c2d42ff8845efd8cb93f2ef P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dff8721755c4283803b31c2e3a0ce2f in term 0.
I20260811 14:35:31.515166 6099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "218a00c37c2d42ff8845efd8cb93f2ef" candidate_uuid: "4dff8721755c4283803b31c2e3a0ce2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" is_pre_election: true
I20260811 14:35:31.515316 6099 raft_consensus.cc:2454] T 218a00c37c2d42ff8845efd8cb93f2ef P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dff8721755c4283803b31c2e3a0ce2f in term 0.
I20260811 14:35:31.515414 5752 leader_election.cc:302] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dff8721755c4283803b31c2e3a0ce2f, fe487f13ca3b45bc8ad4cd08c1e604a8; no voters:
I20260811 14:35:31.515525 6209 raft_consensus.cc:2781] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:35:31.515596 6209 raft_consensus.cc:482] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:35:31.515628 6209 raft_consensus.cc:3037] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:31.516386 6209 raft_consensus.cc:504] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:31.516541 6209 leader_election.cc:288] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [CANDIDATE]: Term 1 election: Requested vote from peers fe487f13ca3b45bc8ad4cd08c1e604a8 (127.2.103.65:34085), b808b4cb2ac343eebfecf1ff12b596f4 (127.2.103.67:46251)
I20260811 14:35:31.516680 5629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "218a00c37c2d42ff8845efd8cb93f2ef" candidate_uuid: "4dff8721755c4283803b31c2e3a0ce2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8"
I20260811 14:35:31.516708 6099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "218a00c37c2d42ff8845efd8cb93f2ef" candidate_uuid: "4dff8721755c4283803b31c2e3a0ce2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b808b4cb2ac343eebfecf1ff12b596f4"
I20260811 14:35:31.516765 5629 raft_consensus.cc:3037] T 218a00c37c2d42ff8845efd8cb93f2ef P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:31.516778 6099 raft_consensus.cc:3037] T 218a00c37c2d42ff8845efd8cb93f2ef P b808b4cb2ac343eebfecf1ff12b596f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:31.517537 6099 raft_consensus.cc:2454] T 218a00c37c2d42ff8845efd8cb93f2ef P b808b4cb2ac343eebfecf1ff12b596f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dff8721755c4283803b31c2e3a0ce2f in term 1.
I20260811 14:35:31.517568 5629 raft_consensus.cc:2454] T 218a00c37c2d42ff8845efd8cb93f2ef P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dff8721755c4283803b31c2e3a0ce2f in term 1.
I20260811 14:35:31.517740 5758 leader_election.cc:302] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dff8721755c4283803b31c2e3a0ce2f, b808b4cb2ac343eebfecf1ff12b596f4; no voters:
I20260811 14:35:31.517854 6209 raft_consensus.cc:2781] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:31.517918 6209 raft_consensus.cc:686] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [term 1 LEADER]: Becoming Leader. State: Replica: 4dff8721755c4283803b31c2e3a0ce2f, State: Running, Role: LEADER
I20260811 14:35:31.517989 6209 consensus_queue.cc:237] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } }
I20260811 14:35:31.518647 5378 catalog_manager.cc:5495] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dff8721755c4283803b31c2e3a0ce2f (127.2.103.66). New cstate: current_term: 1 leader_uuid: "4dff8721755c4283803b31c2e3a0ce2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dff8721755c4283803b31c2e3a0ce2f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 39447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260811 14:35:31.536204 6374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:31.533812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46442 (local address 127.2.103.66:39447)
0811 14:35:31.534003 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:31.534005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:31.534010 (+ 5us) server_negotiation.cc:407] Connection header received
0811 14:35:31.534028 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:31.534028 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:31.534038 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:31.534104 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:31.534231 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:31.534872 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:31.535550 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:31.535822 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:31.535879 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:31.536004 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:31.536034 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:31.536073 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:31.536123 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 14:35:31.536140 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":120,"threads_started":1}
I20260811 14:35:31.538524 6099 raft_consensus.cc:1261] T 218a00c37c2d42ff8845efd8cb93f2ef P b808b4cb2ac343eebfecf1ff12b596f4 [term 1 FOLLOWER]: Refusing update from remote peer 4dff8721755c4283803b31c2e3a0ce2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:31.538642 5629 raft_consensus.cc:1261] T 218a00c37c2d42ff8845efd8cb93f2ef P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 FOLLOWER]: Refusing update from remote peer 4dff8721755c4283803b31c2e3a0ce2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:31.538856 6318 consensus_queue.cc:1035] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b808b4cb2ac343eebfecf1ff12b596f4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:31.538985 6209 consensus_queue.cc:1035] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe487f13ca3b45bc8ad4cd08c1e604a8" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 14:35:31.543753 6304 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 14:35:33.065845 5381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:33.065982 5381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:33.067045 5381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3859fee308d049e5b74f9e85aa2b051b: Sending DeleteTablet for 3 replicas of tablet 218a00c37c2d42ff8845efd8cb93f2ef
I20260811 14:35:33.067329 5604 tablet_service.cc:1514] Processing DeleteTablet for tablet 218a00c37c2d42ff8845efd8cb93f2ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:33 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:42782
I20260811 14:35:33.067346 6067 tablet_service.cc:1514] Processing DeleteTablet for tablet 218a00c37c2d42ff8845efd8cb93f2ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:33 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:43666
I20260811 14:35:33.067346 5836 tablet_service.cc:1514] Processing DeleteTablet for tablet 218a00c37c2d42ff8845efd8cb93f2ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:33 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:46412
I20260811 14:35:33.067569 6414 tablet_replica.cc:330] T 218a00c37c2d42ff8845efd8cb93f2ef P b808b4cb2ac343eebfecf1ff12b596f4: stopping tablet replica
I20260811 14:35:33.067584 6411 tablet_replica.cc:330] T 218a00c37c2d42ff8845efd8cb93f2ef P fe487f13ca3b45bc8ad4cd08c1e604a8: stopping tablet replica
I20260811 14:35:33.067636 6411 raft_consensus.cc:2229] T 218a00c37c2d42ff8845efd8cb93f2ef P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:33.067634 6414 raft_consensus.cc:2229] T 218a00c37c2d42ff8845efd8cb93f2ef P b808b4cb2ac343eebfecf1ff12b596f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:33.067679 6411 raft_consensus.cc:2258] T 218a00c37c2d42ff8845efd8cb93f2ef P fe487f13ca3b45bc8ad4cd08c1e604a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:33.067679 6414 raft_consensus.cc:2258] T 218a00c37c2d42ff8845efd8cb93f2ef P b808b4cb2ac343eebfecf1ff12b596f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:33.067808 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 5502
I20260811 14:35:33.096786 6415 tablet_replica.cc:330] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f: stopping tablet replica
I20260811 14:35:33.096911 6415 raft_consensus.cc:2229] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:35:33.096865 6414 ts_tablet_manager.cc:1905] T 218a00c37c2d42ff8845efd8cb93f2ef P b808b4cb2ac343eebfecf1ff12b596f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:33.097002 6415 raft_consensus.cc:2258] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:33.097327 6415 ts_tablet_manager.cc:1905] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:33.098785 6414 ts_tablet_manager.cc:1918] T 218a00c37c2d42ff8845efd8cb93f2ef P b808b4cb2ac343eebfecf1ff12b596f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 14:35:33.098871 6414 log.cc:1196] T 218a00c37c2d42ff8845efd8cb93f2ef P b808b4cb2ac343eebfecf1ff12b596f4: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/218a00c37c2d42ff8845efd8cb93f2ef
I20260811 14:35:33.099117 6414 ts_tablet_manager.cc:1939] T 218a00c37c2d42ff8845efd8cb93f2ef P b808b4cb2ac343eebfecf1ff12b596f4: Deleting consensus metadata
I20260811 14:35:33.099555 5357 catalog_manager.cc:4841] TS b808b4cb2ac343eebfecf1ff12b596f4 (127.2.103.67:46251): tablet 218a00c37c2d42ff8845efd8cb93f2ef (table test-table [id=68b7e58173e141588d90a7c71e99cd4f]) successfully deleted
I20260811 14:35:33.099622 6415 ts_tablet_manager.cc:1918] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 14:35:33.099699 6415 log.cc:1196] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/218a00c37c2d42ff8845efd8cb93f2ef
I20260811 14:35:33.099936 6415 ts_tablet_manager.cc:1939] T 218a00c37c2d42ff8845efd8cb93f2ef P 4dff8721755c4283803b31c2e3a0ce2f: Deleting consensus metadata
I20260811 14:35:33.100347 5356 catalog_manager.cc:4841] TS 4dff8721755c4283803b31c2e3a0ce2f (127.2.103.66:39447): tablet 218a00c37c2d42ff8845efd8cb93f2ef (table test-table [id=68b7e58173e141588d90a7c71e99cd4f]) successfully deleted
W20260811 14:35:33.103816 5355 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.103.65:34085 (error 108)
I20260811 14:35:33.104113 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 5728
I20260811 14:35:33.107113 6422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:33.104240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:34085 (local address 127.0.0.1:60216)
0811 14:35:33.107010 (+ 2770us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:33.107071 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.65:34085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2738,"thread_start_us":66,"threads_started":1}
W20260811 14:35:33.107717 5355 catalog_manager.cc:4584] TS fe487f13ca3b45bc8ad4cd08c1e604a8 (127.2.103.65:34085): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 218a00c37c2d42ff8845efd8cb93f2ef: Network error: Client connection negotiation failed: client connection to 127.2.103.65:34085: connect: Connection refused (error 111)
I20260811 14:35:33.115981 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 5958
I20260811 14:35:33.136328 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 5325
2026-08-11T14:35:33Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5963 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:33 dist-test-slave-twwt krb5kdc[6426](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:33 dist-test-slave-twwt krb5kdc[6426](info): set up 2 sockets
Aug 11 14:35:33 dist-test-slave-twwt krb5kdc[6426](info): commencing operation
krb5kdc: starting...
W20260811 14:35:34.459878 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.294s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:35:34 dist-test-slave-twwt krb5kdc[6426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:35:34.594413 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:39069
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:43299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:39069
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:34.683485 6447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:34.683771 6447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:34.683817 6447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:34.685105 6447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:34.685189 6447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:34.685213 6447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:34.685237 6447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:34.685266 6447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:34.686976 6447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:43299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:39069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:39069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:34.687302 6447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:34.687573 6447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:34.693421 6456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:34.693439 6458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:34.693468 6447 server_base.cc:1034] running on GCE node
W20260811 14:35:34.693449 6453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:34.694082 6447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:34.694454 6447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:34.695662 6447 hybrid_clock.cc:648] HybridClock initialized: now 1786458934695651 us; error 42 us; skew 500 ppm
Aug 11 14:35:34 dist-test-slave-twwt krb5kdc[6426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458934, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:34.702363 6447 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:35:34.702947 6447 webserver.cc:460] Webserver started at http://127.2.103.126:38643/ using document root <none> and password file <none>
I20260811 14:35:34.703202 6447 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:34.703275 6447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:34.703464 6447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:34.704617 6447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "98fd1bfd14de4732968f73be6c545234"
format_stamp: "Formatted at 2026-08-11 14:35:34 on dist-test-slave-twwt"
server_key: "cea04c6f433dac031cd94a44a1b46e61"
server_key_iv: "75f5d75d9b16d79cddb385fedc0278fb"
server_key_version: "encryptionkey@0"
I20260811 14:35:34.705096 6447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "98fd1bfd14de4732968f73be6c545234"
format_stamp: "Formatted at 2026-08-11 14:35:34 on dist-test-slave-twwt"
server_key: "cea04c6f433dac031cd94a44a1b46e61"
server_key_iv: "75f5d75d9b16d79cddb385fedc0278fb"
server_key_version: "encryptionkey@0"
I20260811 14:35:34.706655 6447 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:34.707535 6471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:34.707793 6447 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:34.707918 6447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "98fd1bfd14de4732968f73be6c545234"
format_stamp: "Formatted at 2026-08-11 14:35:34 on dist-test-slave-twwt"
server_key: "cea04c6f433dac031cd94a44a1b46e61"
server_key_iv: "75f5d75d9b16d79cddb385fedc0278fb"
server_key_version: "encryptionkey@0"
I20260811 14:35:34.708005 6447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:34.719144 6447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:34.721199 6447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:34.721388 6447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:34.725782 6447 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:39069
I20260811 14:35:34.725836 6563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:39069 every 8 connection(s)
I20260811 14:35:34.726239 6447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:34.727268 6565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:34.729686 6565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234: Bootstrap starting.
I20260811 14:35:34.730306 6565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:34.730571 6565 log.cc:824] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:34.730908 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 6447
I20260811 14:35:34.731024 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:35:34.731244 2461 external_mini_cluster.cc:1428] Setting key e48a66456917862936f3606e8b9e444b
I20260811 14:35:34.731426 6565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234: No bootstrap required, opened a new log
I20260811 14:35:34.733354 6565 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98fd1bfd14de4732968f73be6c545234" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 39069 } }
I20260811 14:35:34.733546 6565 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:34.733587 6565 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98fd1bfd14de4732968f73be6c545234, State: Initialized, Role: FOLLOWER
I20260811 14:35:34.733780 6565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98fd1bfd14de4732968f73be6c545234" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 39069 } }
I20260811 14:35:34.733887 6565 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:34.733932 6565 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:34.733990 6565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:34.735131 6565 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98fd1bfd14de4732968f73be6c545234" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 39069 } }
I20260811 14:35:34.735234 6565 leader_election.cc:302] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98fd1bfd14de4732968f73be6c545234; no voters:
I20260811 14:35:34.735455 6565 leader_election.cc:288] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:34.735487 6574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:34.735599 6574 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234 [term 1 LEADER]: Becoming Leader. State: Replica: 98fd1bfd14de4732968f73be6c545234, State: Running, Role: LEADER
I20260811 14:35:34.735757 6565 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:34.735766 6574 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98fd1bfd14de4732968f73be6c545234" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 39069 } }
I20260811 14:35:34.736172 6574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98fd1bfd14de4732968f73be6c545234. Latest consensus state: current_term: 1 leader_uuid: "98fd1bfd14de4732968f73be6c545234" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98fd1bfd14de4732968f73be6c545234" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 39069 } } }
I20260811 14:35:34.736389 6574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:34.736188 6575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98fd1bfd14de4732968f73be6c545234" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98fd1bfd14de4732968f73be6c545234" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 39069 } } }
I20260811 14:35:34.736624 6575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:34.736763 6586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:34.737329 6586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:34.739319 6586 catalog_manager.cc:1347] Generated new cluster ID: a9fb4ddb691447aa98db5fdd959f8c23
I20260811 14:35:34.739372 6586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:35:34 dist-test-slave-twwt krb5kdc[6426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:34.746116 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:34.731894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58138 (local address 127.2.103.126:39069)
0811 14:35:34.732101 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:34.732114 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:34.732148 (+ 34us) server_negotiation.cc:407] Connection header received
0811 14:35:34.733338 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:34.733348 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:34.733413 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:34.733588 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:34.734462 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:34.735908 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:34.737187 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:34.738004 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:34.738475 (+ 471us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:34.741422 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:34.741426 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:34.741426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:34.741438 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:34.743686 (+ 2248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:34.744192 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:34.744196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:34.744196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:34.744730 (+ 534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:34.745018 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:34.745019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:34.745020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:34.745383 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:34.745552 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:34.745735 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 14:35:34.745891 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260811 14:35:34.750644 6586 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:34.751480 6586 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:34.760979 6586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98fd1bfd14de4732968f73be6c545234: Generated new TSK 0
I20260811 14:35:34.761284 6586 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
I20260811 14:35:34.799880 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:39069
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:43299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:34.886490 6604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:34.886801 6604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:34.886847 6604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:34.888063 6604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:34.888139 6604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:34.888187 6604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:35:34.889863 6604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:43299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:39069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:34.890151 6604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:34.890389 6604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:34.895759 6615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:34.895748 6609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:34.895752 6610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:34.895793 6604 server_base.cc:1034] running on GCE node
I20260811 14:35:34.896222 6604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:34.896515 6604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:34.897691 6604 hybrid_clock.cc:648] HybridClock initialized: now 1786458934897684 us; error 36 us; skew 500 ppm
Aug 11 14:35:34 dist-test-slave-twwt krb5kdc[6426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458934, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:34.900403 6604 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:35:34.900880 6604 webserver.cc:460] Webserver started at http://127.2.103.65:33677/ using document root <none> and password file <none>
I20260811 14:35:34.901120 6604 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:34.901189 6604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:34.901352 6604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:34.902452 6604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "b7aea10e8cb44ea19d07d2fee852b494"
format_stamp: "Formatted at 2026-08-11 14:35:34 on dist-test-slave-twwt"
server_key: "66733c2264a219faa3a71afb97d96e6a"
server_key_iv: "8be2deedc38a568ad85386c6a6f68efc"
server_key_version: "encryptionkey@0"
I20260811 14:35:34.902858 6604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "b7aea10e8cb44ea19d07d2fee852b494"
format_stamp: "Formatted at 2026-08-11 14:35:34 on dist-test-slave-twwt"
server_key: "66733c2264a219faa3a71afb97d96e6a"
server_key_iv: "8be2deedc38a568ad85386c6a6f68efc"
server_key_version: "encryptionkey@0"
I20260811 14:35:34.904253 6604 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:35:34.905018 6624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:34.905195 6604 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 14:35:34.905318 6604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "b7aea10e8cb44ea19d07d2fee852b494"
format_stamp: "Formatted at 2026-08-11 14:35:34 on dist-test-slave-twwt"
server_key: "66733c2264a219faa3a71afb97d96e6a"
server_key_iv: "8be2deedc38a568ad85386c6a6f68efc"
server_key_version: "encryptionkey@0"
I20260811 14:35:34.905396 6604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:34.919727 6604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:34.921782 6604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:34.921959 6604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:34.922201 6604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:34.922549 6604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:34.922605 6604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:34.922672 6604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:34.922698 6604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:34.928787 6604 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:44977
I20260811 14:35:34.928869 6804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:44977 every 8 connection(s)
I20260811 14:35:34.929206 6604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:35:34 dist-test-slave-twwt krb5kdc[6426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458934, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:34.936435 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 6604
I20260811 14:35:34.936540 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:35:34.936753 2461 external_mini_cluster.cc:1428] Setting key 4c5916084e8833d0898d30d1bdf34440
I20260811 14:35:34.939188 6807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:34.929999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:39069 (local address 127.2.103.65:49547)
0811 14:35:34.930173 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:34.930235 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:34.930803 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:34.931027 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:34.931030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:34.931617 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:34.931815 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:34.931820 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:34.932490 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:34.932491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:34.933306 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:34.933308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:34.933682 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:34.933683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:34.933785 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:34.934344 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:34.934353 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:34.935734 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:34.937791 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:34.937798 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:34.937800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:34.938027 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:34.938252 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:34.938253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:34.938253 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:34.938355 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:34.938666 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:34.938673 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:34.938805 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 14:35:34.938894 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 14:35:34.939023 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260811 14:35:34.939185 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:34.930041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:49547 (local address 127.2.103.126:39069)
0811 14:35:34.930126 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:34.930128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:34.930333 (+ 205us) server_negotiation.cc:407] Connection header received
0811 14:35:34.930894 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:34.930896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:34.930910 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:34.930972 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:34.931900 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:34.932409 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:34.933392 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:34.933602 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:34.933659 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:34.935872 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:34.935874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:34.935874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:34.935883 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:34.937708 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:34.938108 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:34.938110 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:34.938110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:34.938175 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:34.938421 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:34.938422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:34.938422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:34.938579 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:34.938666 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:34.938940 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 14:35:34.939042 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 14:35:34.940131 6805 heartbeater.cc:347] Connected to a master server at 127.2.103.126:39069
I20260811 14:35:34.940249 6805 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:34.940438 6805 heartbeater.cc:511] Master 127.2.103.126:39069 requested a full tablet report, sending...
I20260811 14:35:34.941057 6501 ts_manager.cc:194] Registered new tserver with Master: b7aea10e8cb44ea19d07d2fee852b494 (127.2.103.65:44977)
I20260811 14:35:34.941684 6501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:49547
WARNING: no policy specified for kudu/127.2.103.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
I20260811 14:35:34.966776 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:39069
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:43299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:35.051573 6812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:35.051831 6812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:35.051885 6812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:35.053092 6812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:35.053164 6812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:35.053210 6812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:35:35.054728 6812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:43299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:39069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:35.055023 6812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:35.055275 6812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:35.060644 6822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:35.060636 6828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:35.060665 6812 server_base.cc:1034] running on GCE node
W20260811 14:35:35.060636 6825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:35.061091 6812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:35.061390 6812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:35.062552 6812 hybrid_clock.cc:648] HybridClock initialized: now 1786458935062548 us; error 36 us; skew 500 ppm
Aug 11 14:35:35 dist-test-slave-twwt krb5kdc[6426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458935, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:35.065176 6812 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:35:35.065649 6812 webserver.cc:460] Webserver started at http://127.2.103.66:40315/ using document root <none> and password file <none>
I20260811 14:35:35.065919 6812 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:35.065999 6812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:35.066183 6812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:35.067410 6812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "6a6ed8e1e27340c3812e82575d5b351a"
format_stamp: "Formatted at 2026-08-11 14:35:35 on dist-test-slave-twwt"
server_key: "823e09aa4d5810972617244269a0f3f4"
server_key_iv: "da11007d03b0fdf56003cc25033fcc2f"
server_key_version: "encryptionkey@0"
I20260811 14:35:35.067791 6812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "6a6ed8e1e27340c3812e82575d5b351a"
format_stamp: "Formatted at 2026-08-11 14:35:35 on dist-test-slave-twwt"
server_key: "823e09aa4d5810972617244269a0f3f4"
server_key_iv: "da11007d03b0fdf56003cc25033fcc2f"
server_key_version: "encryptionkey@0"
I20260811 14:35:35.069186 6812 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:35:35.070014 6841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:35.070225 6812 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:35.070335 6812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "6a6ed8e1e27340c3812e82575d5b351a"
format_stamp: "Formatted at 2026-08-11 14:35:35 on dist-test-slave-twwt"
server_key: "823e09aa4d5810972617244269a0f3f4"
server_key_iv: "da11007d03b0fdf56003cc25033fcc2f"
server_key_version: "encryptionkey@0"
I20260811 14:35:35.070413 6812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:35.082569 6812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:35.084555 6812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:35.084739 6812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:35.084996 6812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:35.085341 6812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:35.085398 6812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:35.085441 6812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:35.085469 6812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:35.091585 6812 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:42137
I20260811 14:35:35.091624 7029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:42137 every 8 connection(s)
I20260811 14:35:35.091982 6812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:35:35.092939 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 6812
I20260811 14:35:35.093051 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:35:35.093264 2461 external_mini_cluster.cc:1428] Setting key a814238067723abd0c3d0e68438ad9de
Aug 11 14:35:35 dist-test-slave-twwt krb5kdc[6426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458935, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:35.102922 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:35.092761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:38103 (local address 127.2.103.126:39069)
0811 14:35:35.092856 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:35.092859 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:35.093025 (+ 166us) server_negotiation.cc:407] Connection header received
0811 14:35:35.093721 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:35.093722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:35.093731 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:35.093767 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:35.095133 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:35.095641 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:35.096769 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:35.097110 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:35.097186 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:35.099828 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:35.099829 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:35.099829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:35.099836 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:35.101347 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:35.101860 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:35.101861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:35.101862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:35.101914 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:35.102214 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:35.102214 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:35.102214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:35.102314 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:35.102367 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:35.102765 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 14:35:35.102839 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
WARNING: no policy specified for kudu/127.2.103.67@KRBTEST.COM; defaulting to no policy
I20260811 14:35:35.103185 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:35.092689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:39069 (local address 127.2.103.66:38103)
0811 14:35:35.092882 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:35.092937 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:35.093621 (+ 684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:35.093852 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:35.093856 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:35.094749 (+ 893us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:35.095042 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:35.095049 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:35.095739 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:35.095740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:35.096677 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:35.096679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:35.097199 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:35.097199 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:35.097305 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:35.097854 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:35.097868 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:35.099730 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:35.101439 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:35.101447 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:35.101450 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:35.101769 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:35.101987 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:35.101988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:35.101988 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:35.102141 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:35.102385 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:35.102393 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:35.102580 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 14:35:35.102733 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 14:35:35.102913 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":96,"threads_started":1}
I20260811 14:35:35.103735 7030 heartbeater.cc:347] Connected to a master server at 127.2.103.126:39069
I20260811 14:35:35.103834 7030 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:35.104043 7030 heartbeater.cc:511] Master 127.2.103.126:39069 requested a full tablet report, sending...
I20260811 14:35:35.104514 6501 ts_manager.cc:194] Registered new tserver with Master: 6a6ed8e1e27340c3812e82575d5b351a (127.2.103.66:42137)
I20260811 14:35:35.104853 6501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:38103
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
I20260811 14:35:35.123950 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:39069
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:43299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:35.210724 7041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:35.211027 7041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:35.211059 7041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:35.212402 7041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:35.212461 7041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:35.212491 7041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:35:35.214025 7041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:43299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:39069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:35.214264 7041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:35.214491 7041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:35.219882 7053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:35.219887 7050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:35.219892 7051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:35.220172 7041 server_base.cc:1034] running on GCE node
I20260811 14:35:35.220417 7041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:35.220722 7041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:35.221915 7041 hybrid_clock.cc:648] HybridClock initialized: now 1786458935221878 us; error 44 us; skew 500 ppm
Aug 11 14:35:35 dist-test-slave-twwt krb5kdc[6426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458935, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:35.224910 7041 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:35:35.225492 7041 webserver.cc:460] Webserver started at http://127.2.103.67:40185/ using document root <none> and password file <none>
I20260811 14:35:35.225756 7041 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:35.225827 7041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:35.225996 7041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:35.227213 7041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "f14f09986b554a00888b3ba271998eee"
format_stamp: "Formatted at 2026-08-11 14:35:35 on dist-test-slave-twwt"
server_key: "727e487764e2b163132d55cb63e65b1d"
server_key_iv: "57134b567d06dc65fad9905305206313"
server_key_version: "encryptionkey@0"
I20260811 14:35:35.227617 7041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "f14f09986b554a00888b3ba271998eee"
format_stamp: "Formatted at 2026-08-11 14:35:35 on dist-test-slave-twwt"
server_key: "727e487764e2b163132d55cb63e65b1d"
server_key_iv: "57134b567d06dc65fad9905305206313"
server_key_version: "encryptionkey@0"
I20260811 14:35:35.229058 7041 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 14:35:35.229888 7064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:35.230069 7041 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:35.230185 7041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "f14f09986b554a00888b3ba271998eee"
format_stamp: "Formatted at 2026-08-11 14:35:35 on dist-test-slave-twwt"
server_key: "727e487764e2b163132d55cb63e65b1d"
server_key_iv: "57134b567d06dc65fad9905305206313"
server_key_version: "encryptionkey@0"
I20260811 14:35:35.230262 7041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:35.242725 7041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:35.244706 7041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:35.244894 7041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:35.245157 7041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:35.245527 7041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:35.245589 7041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:35.245637 7041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:35.245664 7041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:35.253556 7041 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:39321
I20260811 14:35:35.253597 7235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:39321 every 8 connection(s)
I20260811 14:35:35.253999 7041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:35:35.260264 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 7041
I20260811 14:35:35.260376 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:35:35.260591 2461 external_mini_cluster.cc:1428] Setting key 5854625d4ec89b4939077fe149cc7137
Aug 11 14:35:35 dist-test-slave-twwt krb5kdc[6426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458935, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:35.264899 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:35.254924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:48967 (local address 127.2.103.126:39069)
0811 14:35:35.254971 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:35.254973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:35.255199 (+ 226us) server_negotiation.cc:407] Connection header received
0811 14:35:35.255696 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:35.255697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:35.255708 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:35.255767 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:35.257104 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:35.257682 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:35.259118 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:35.259426 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:35.259506 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:35.262265 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:35.262267 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:35.262268 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:35.262275 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:35.263603 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:35.263987 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:35.263988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:35.263988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:35.264044 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:35.264298 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:35.264299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:35.264299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:35.264407 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:35.264460 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:35.264717 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 14:35:35.264829 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:35:35.265091 7240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:35.254887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:39069 (local address 127.2.103.67:48967)
0811 14:35:35.255061 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:35.255107 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:35.255604 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:35.255833 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:35.255836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:35.256621 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:35.256975 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:35.256982 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:35.257790 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:35.257791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:35.259028 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:35.259031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:35.259529 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:35.259530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:35.259680 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:35.260258 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:35.260271 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:35.262162 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:35.263676 (+ 1514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:35.263683 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:35.263685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:35.263916 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:35.264129 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:35.264130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:35.264130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:35.264239 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:35.264466 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:35.264471 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:35.264593 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 14:35:35.264679 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 14:35:35.264867 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
I20260811 14:35:35.265684 7237 heartbeater.cc:347] Connected to a master server at 127.2.103.126:39069
I20260811 14:35:35.265779 7237 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:35.265954 7237 heartbeater.cc:511] Master 127.2.103.126:39069 requested a full tablet report, sending...
I20260811 14:35:35.266305 6501 ts_manager.cc:194] Registered new tserver with Master: f14f09986b554a00888b3ba271998eee (127.2.103.67:39321)
I20260811 14:35:35.266789 6501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:48967
I20260811 14:35:35.272562 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 14:35:35.286664 6573 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:35.285461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58150 (local address 127.2.103.126:39069)
0811 14:35:35.285584 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:35.285588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:35.285610 (+ 22us) server_negotiation.cc:407] Connection header received
0811 14:35:35.285730 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:35.285732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:35.285751 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:35.285827 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:35.286536 (+ 709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58150: BlockingRecv error: recv got EOF from 127.0.0.1:58150 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 14:35:35.288129 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 6604
I20260811 14:35:35.317305 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 6812
I20260811 14:35:35.350917 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 7041
I20260811 14:35:35.367211 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 6447
2026-08-11T14:35:35Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2227 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:35 dist-test-slave-twwt krb5kdc[7255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:35 dist-test-slave-twwt krb5kdc[7255](info): set up 2 sockets
Aug 11 14:35:35 dist-test-slave-twwt krb5kdc[7255](info): commencing operation
krb5kdc: starting...
W20260811 14:35:36.650352 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:35:36 dist-test-slave-twwt krb5kdc[7255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:35:36.781515 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:43307
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:38159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:43307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:36.868495 7273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:36.868768 7273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:36.868810 7273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:36.870013 7273 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:36.870090 7273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:36.870114 7273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:36.870136 7273 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:36.870167 7273 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:36.871779 7273 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:38159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:43307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:43307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:36.872087 7273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:36.872334 7273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:36.877646 7281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:36.877650 7280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:36.877770 7273 server_base.cc:1034] running on GCE node
W20260811 14:35:36.877627 7283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:36.878111 7273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:36.878451 7273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:36.879601 7273 hybrid_clock.cc:648] HybridClock initialized: now 1786458936879579 us; error 43 us; skew 500 ppm
Aug 11 14:35:36 dist-test-slave-twwt krb5kdc[7255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458936, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:36.882234 7273 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:35:36.882862 7273 webserver.cc:460] Webserver started at http://127.2.103.126:33261/ using document root <none> and password file <none>
I20260811 14:35:36.883106 7273 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:36.883178 7273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:36.883347 7273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:36.884462 7273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "7bf44e55ee764d34bd4ab39c5ec33bee"
format_stamp: "Formatted at 2026-08-11 14:35:36 on dist-test-slave-twwt"
server_key: "3a0f62692f9a882873bf4dd09045d100"
server_key_iv: "3af8333924316306b8757db22bc7f0a8"
server_key_version: "encryptionkey@0"
I20260811 14:35:36.884862 7273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "7bf44e55ee764d34bd4ab39c5ec33bee"
format_stamp: "Formatted at 2026-08-11 14:35:36 on dist-test-slave-twwt"
server_key: "3a0f62692f9a882873bf4dd09045d100"
server_key_iv: "3af8333924316306b8757db22bc7f0a8"
server_key_version: "encryptionkey@0"
I20260811 14:35:36.886274 7273 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:36.887189 7296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:36.887400 7273 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:35:36.887518 7273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "7bf44e55ee764d34bd4ab39c5ec33bee"
format_stamp: "Formatted at 2026-08-11 14:35:36 on dist-test-slave-twwt"
server_key: "3a0f62692f9a882873bf4dd09045d100"
server_key_iv: "3af8333924316306b8757db22bc7f0a8"
server_key_version: "encryptionkey@0"
I20260811 14:35:36.887603 7273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:36.897928 7273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:36.900204 7273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:36.900398 7273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:36.904979 7273 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:43307
I20260811 14:35:36.905030 7380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:43307 every 8 connection(s)
I20260811 14:35:36.905453 7273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:36.906405 7381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:36.907821 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 7273
I20260811 14:35:36.907935 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:35:36.908109 2461 external_mini_cluster.cc:1428] Setting key 1025484305b0a202599567faba6ffb2a
I20260811 14:35:36.909322 7381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee: Bootstrap starting.
I20260811 14:35:36.910172 7381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:36.910528 7381 log.cc:824] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:36.911367 7381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee: No bootstrap required, opened a new log
I20260811 14:35:36.913913 7381 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bf44e55ee764d34bd4ab39c5ec33bee" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 43307 } }
I20260811 14:35:36.914108 7381 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:36.914151 7381 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bf44e55ee764d34bd4ab39c5ec33bee, State: Initialized, Role: FOLLOWER
I20260811 14:35:36.914314 7381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bf44e55ee764d34bd4ab39c5ec33bee" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 43307 } }
I20260811 14:35:36.914409 7381 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:36.914448 7381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:36.914508 7381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:36.915282 7381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bf44e55ee764d34bd4ab39c5ec33bee" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 43307 } }
I20260811 14:35:36.915380 7381 leader_election.cc:302] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7bf44e55ee764d34bd4ab39c5ec33bee; no voters:
Aug 11 14:35:36 dist-test-slave-twwt krb5kdc[7255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:36.915576 7381 leader_election.cc:288] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:36.915649 7388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:36.915809 7388 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee [term 1 LEADER]: Becoming Leader. State: Replica: 7bf44e55ee764d34bd4ab39c5ec33bee, State: Running, Role: LEADER
I20260811 14:35:36.915884 7381 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:36.916014 7388 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bf44e55ee764d34bd4ab39c5ec33bee" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 43307 } }
I20260811 14:35:36.916512 7391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7bf44e55ee764d34bd4ab39c5ec33bee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bf44e55ee764d34bd4ab39c5ec33bee" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 43307 } } }
I20260811 14:35:36.916591 7391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:36.916891 7392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bf44e55ee764d34bd4ab39c5ec33bee. Latest consensus state: current_term: 1 leader_uuid: "7bf44e55ee764d34bd4ab39c5ec33bee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bf44e55ee764d34bd4ab39c5ec33bee" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 43307 } } }
I20260811 14:35:36.917028 7392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:36.916962 7400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:36.917598 7400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:36.919462 7400 catalog_manager.cc:1347] Generated new cluster ID: 6939cbb9c0cb4db4a62f485fe442b788
I20260811 14:35:36.919519 7400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:35:36.920876 7386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:36.908671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47300 (local address 127.2.103.126:43307)
0811 14:35:36.908869 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:36.908877 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:36.908904 (+ 27us) server_negotiation.cc:407] Connection header received
0811 14:35:36.909305 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:36.909313 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:36.909365 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:36.909535 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:36.910394 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:36.911571 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:36.912643 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:36.913071 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:36.913156 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:36.916188 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:36.916192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:36.916192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:36.916206 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:36.919185 (+ 2979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:36.919677 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:36.919680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:36.919681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:36.919784 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:36.920006 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:36.920007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:36.920007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:36.920264 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:36.920409 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:36.920534 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 14:35:36.920681 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
I20260811 14:35:36.921823 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:35:36.925750 7400 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:36.926311 7400 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:36.928861 7400 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7bf44e55ee764d34bd4ab39c5ec33bee: Generated new TSK 0
I20260811 14:35:36.929054 7400 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:35:36.941628 7386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:36.937520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47316 (local address 127.2.103.126:43307)
0811 14:35:36.937590 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:36.937592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:36.937686 (+ 94us) server_negotiation.cc:407] Connection header received
0811 14:35:36.937771 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:36.937772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:36.937785 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:36.937844 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:36.938246 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:36.939073 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:36.939884 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:36.940212 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:36.940287 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:36.941062 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:36.941063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:36.941064 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:36.941326 (+ 262us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:36.941353 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:36.941361 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:36.941424 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:36.941506 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 14:35:36.941539 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 14:35:36.945165 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 7273
2026-08-11T14:35:36Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1580 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:37 dist-test-slave-twwt krb5kdc[7433](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:37 dist-test-slave-twwt krb5kdc[7433](info): set up 2 sockets
Aug 11 14:35:37 dist-test-slave-twwt krb5kdc[7433](info): commencing operation
krb5kdc: starting...
W20260811 14:35:38.224689 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:35:38 dist-test-slave-twwt krb5kdc[7433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:35:38.359452 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:43905
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:33653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:43905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:38.445605 7451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:38.445914 7451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:38.445959 7451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:38.447219 7451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:38.447289 7451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:38.447319 7451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:38.447340 7451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:38.447371 7451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:38.448982 7451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:33653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:43905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:43905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:38.449301 7451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:38.449558 7451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:38.454581 7464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:38.454591 7462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:38.454587 7461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:38.454993 7451 server_base.cc:1034] running on GCE node
I20260811 14:35:38.455235 7451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:38.455482 7451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:38.456650 7451 hybrid_clock.cc:648] HybridClock initialized: now 1786458938456638 us; error 39 us; skew 500 ppm
Aug 11 14:35:38 dist-test-slave-twwt krb5kdc[7433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458938, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:38.459246 7451 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:35:38.459685 7451 webserver.cc:460] Webserver started at http://127.2.103.126:40651/ using document root <none> and password file <none>
I20260811 14:35:38.459936 7451 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:38.460006 7451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:38.460157 7451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:38.461261 7451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "8e9c2637b83443d4a5650703096e25f4"
format_stamp: "Formatted at 2026-08-11 14:35:38 on dist-test-slave-twwt"
server_key: "267fb4ca5b15d540980c9b5ecf749d35"
server_key_iv: "2b31620635f6d44fae80f34066b1c851"
server_key_version: "encryptionkey@0"
I20260811 14:35:38.461655 7451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "8e9c2637b83443d4a5650703096e25f4"
format_stamp: "Formatted at 2026-08-11 14:35:38 on dist-test-slave-twwt"
server_key: "267fb4ca5b15d540980c9b5ecf749d35"
server_key_iv: "2b31620635f6d44fae80f34066b1c851"
server_key_version: "encryptionkey@0"
I20260811 14:35:38.463124 7451 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:35:38.463878 7480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:38.464057 7451 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 14:35:38.464174 7451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "8e9c2637b83443d4a5650703096e25f4"
format_stamp: "Formatted at 2026-08-11 14:35:38 on dist-test-slave-twwt"
server_key: "267fb4ca5b15d540980c9b5ecf749d35"
server_key_iv: "2b31620635f6d44fae80f34066b1c851"
server_key_version: "encryptionkey@0"
I20260811 14:35:38.464257 7451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:38.479182 7451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:38.481343 7451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:38.481479 7451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:38.485771 7451 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:43905
I20260811 14:35:38.485885 7561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:43905 every 8 connection(s)
I20260811 14:35:38.486220 7451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:38.487545 7562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:38.489982 7562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4: Bootstrap starting.
I20260811 14:35:38.490651 7562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:38.490967 7562 log.cc:824] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:38.491806 7562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4: No bootstrap required, opened a new log
I20260811 14:35:38.493513 7562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9c2637b83443d4a5650703096e25f4" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 43905 } }
I20260811 14:35:38.493685 7562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:38.493736 7562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9c2637b83443d4a5650703096e25f4, State: Initialized, Role: FOLLOWER
I20260811 14:35:38.493903 7562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9c2637b83443d4a5650703096e25f4" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 43905 } }
I20260811 14:35:38.493994 7562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:38.494040 7562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:38.494092 7562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:38.495033 7562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9c2637b83443d4a5650703096e25f4" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 43905 } }
I20260811 14:35:38.495134 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 7451
I20260811 14:35:38.495167 7562 leader_election.cc:302] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e9c2637b83443d4a5650703096e25f4; no voters:
I20260811 14:35:38.495285 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:35:38.495406 7562 leader_election.cc:288] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:38.495438 7566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:38.495512 2461 external_mini_cluster.cc:1428] Setting key 0c559ee0713fff6ab226b174e55eb71f
I20260811 14:35:38.495563 7566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9c2637b83443d4a5650703096e25f4, State: Running, Role: LEADER
I20260811 14:35:38.495642 7566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9c2637b83443d4a5650703096e25f4" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 43905 } }
I20260811 14:35:38.495703 7562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:38.495954 7566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e9c2637b83443d4a5650703096e25f4. Latest consensus state: current_term: 1 leader_uuid: "8e9c2637b83443d4a5650703096e25f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9c2637b83443d4a5650703096e25f4" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 43905 } } }
I20260811 14:35:38.496045 7568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e9c2637b83443d4a5650703096e25f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9c2637b83443d4a5650703096e25f4" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 43905 } } }
I20260811 14:35:38.496096 7566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:38.496177 7568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:38.496444 7573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:38.497680 7573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:38.499483 7573 catalog_manager.cc:1347] Generated new cluster ID: 5d69fcaa159645b096a1b00f952d2d9d
I20260811 14:35:38.499540 7573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:35:38 dist-test-slave-twwt krb5kdc[7433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:38.508001 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:38.496077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34472 (local address 127.2.103.126:43905)
0811 14:35:38.496356 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:38.496367 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:38.496406 (+ 39us) server_negotiation.cc:407] Connection header received
0811 14:35:38.496538 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:38.496546 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:38.496607 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:38.496753 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:38.497619 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:38.499006 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:38.500152 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:38.500784 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:38.500993 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:38.503858 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:38.503863 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:38.503863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:38.503878 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:38.506137 (+ 2259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:38.506578 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:38.506582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:38.506582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:38.506748 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:38.506972 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:38.506973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:38.506973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:38.507347 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:38.507525 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:38.507612 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 14:35:38.507763 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":99,"threads_started":1}
I20260811 14:35:38.520851 7573 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:38.521633 7573 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:38.531991 7573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e9c2637b83443d4a5650703096e25f4: Generated new TSK 0
I20260811 14:35:38.532256 7573 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
I20260811 14:35:38.567279 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:43905
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:33653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:38.659098 7602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:38.659372 7602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:38.659420 7602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:38.660696 7602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:38.660770 7602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:38.660818 7602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:35:38.662458 7602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:33653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:43905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:38.662784 7602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:38.663053 7602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:38.668604 7612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:38.668604 7619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:38.668624 7615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:38.669015 7602 server_base.cc:1034] running on GCE node
I20260811 14:35:38.669229 7602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:38.669456 7602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:38.670660 7602 hybrid_clock.cc:648] HybridClock initialized: now 1786458938670593 us; error 27 us; skew 500 ppm
Aug 11 14:35:38 dist-test-slave-twwt krb5kdc[7433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458938, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:38.673573 7602 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:35:38.674081 7602 webserver.cc:460] Webserver started at http://127.2.103.65:44755/ using document root <none> and password file <none>
I20260811 14:35:38.674332 7602 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:38.674405 7602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:38.674569 7602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:38.675715 7602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "9cf51ec3e57745a1a50503587c990fef"
format_stamp: "Formatted at 2026-08-11 14:35:38 on dist-test-slave-twwt"
server_key: "04b1e64069a29990e55acd1722fb7582"
server_key_iv: "bcd933f01bd5939ad3b2ebe44b4af68d"
server_key_version: "encryptionkey@0"
I20260811 14:35:38.676174 7602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "9cf51ec3e57745a1a50503587c990fef"
format_stamp: "Formatted at 2026-08-11 14:35:38 on dist-test-slave-twwt"
server_key: "04b1e64069a29990e55acd1722fb7582"
server_key_iv: "bcd933f01bd5939ad3b2ebe44b4af68d"
server_key_version: "encryptionkey@0"
I20260811 14:35:38.677585 7602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:38.678406 7631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:38.678653 7602 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:38.678769 7602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "9cf51ec3e57745a1a50503587c990fef"
format_stamp: "Formatted at 2026-08-11 14:35:38 on dist-test-slave-twwt"
server_key: "04b1e64069a29990e55acd1722fb7582"
server_key_iv: "bcd933f01bd5939ad3b2ebe44b4af68d"
server_key_version: "encryptionkey@0"
I20260811 14:35:38.678857 7602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:38.706393 7602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:38.709023 7602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:38.709224 7602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:38.709522 7602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:38.709878 7602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:38.709940 7602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:38.709977 7602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:38.710024 7602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:38.715873 7602 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:40373
I20260811 14:35:38.715919 7813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:40373 every 8 connection(s)
I20260811 14:35:38.716300 7602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:35:38 dist-test-slave-twwt krb5kdc[7433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458938, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:38.724665 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 7602
I20260811 14:35:38.724799 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:35:38.725008 2461 external_mini_cluster.cc:1428] Setting key 2e9bcc6a4388b3bacf70e73d08d15fa8
I20260811 14:35:38.726534 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:38.717246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:57547 (local address 127.2.103.126:43905)
0811 14:35:38.717348 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:38.717351 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:38.717558 (+ 207us) server_negotiation.cc:407] Connection header received
0811 14:35:38.718184 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:38.718186 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:38.718203 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:38.718256 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:38.719333 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:38.719849 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:38.720884 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:38.721136 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:38.721218 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:38.723488 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:38.723489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:38.723490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:38.723497 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:38.725009 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:38.725508 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:38.725509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:38.725510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:38.725574 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:38.725855 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:38.725857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:38.725857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:38.725979 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:38.726053 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:38.726347 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 14:35:38.726442 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 14:35:38.726797 7816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:38.717198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:43905 (local address 127.2.103.65:57547)
0811 14:35:38.717410 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:38.717477 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:38.718058 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:38.718336 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:38.718340 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:38.719020 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:38.719224 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:38.719229 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:38.719964 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:38.719967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:38.720787 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:38.720789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:38.721227 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:38.721228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:38.721355 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:38.721902 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:38.721910 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:38.723394 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:38.725128 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:38.725136 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:38.725139 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:38.725417 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:38.725647 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:38.725648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:38.725649 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:38.725770 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:38.726058 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:38.726065 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:38.726209 (+ 144us) client_negotiation.cc:769] Sending connection context
0811 14:35:38.726322 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 14:35:38.726482 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":108,"threads_started":1}
I20260811 14:35:38.727288 7814 heartbeater.cc:347] Connected to a master server at 127.2.103.126:43905
I20260811 14:35:38.727384 7814 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:38.727591 7814 heartbeater.cc:511] Master 127.2.103.126:43905 requested a full tablet report, sending...
I20260811 14:35:38.728212 7509 ts_manager.cc:194] Registered new tserver with Master: 9cf51ec3e57745a1a50503587c990fef (127.2.103.65:40373)
I20260811 14:35:38.728919 7509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:57547
WARNING: no policy specified for kudu/127.2.103.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
I20260811 14:35:38.755893 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:43905
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:33653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:38.840772 7822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:38.841048 7822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:38.841090 7822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:38.842321 7822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:38.842402 7822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:38.842450 7822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:35:38.844148 7822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:33653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:43905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:38.844462 7822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:38.844723 7822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:38.849929 7833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:38.850014 7831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:38.850059 7830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:38.850347 7822 server_base.cc:1034] running on GCE node
I20260811 14:35:38.850759 7822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:38.851050 7822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:38.852250 7822 hybrid_clock.cc:648] HybridClock initialized: now 1786458938852227 us; error 32 us; skew 500 ppm
Aug 11 14:35:38 dist-test-slave-twwt krb5kdc[7433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458938, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:38.854981 7822 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:35:38.855494 7822 webserver.cc:460] Webserver started at http://127.2.103.66:38125/ using document root <none> and password file <none>
I20260811 14:35:38.855757 7822 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:38.855829 7822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:38.855980 7822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:38.857052 7822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "91155c7391d1433ba68b5998872412c0"
format_stamp: "Formatted at 2026-08-11 14:35:38 on dist-test-slave-twwt"
server_key: "c43886253bfd9471038ba2d38634280f"
server_key_iv: "68e0def97dd11cbe5e9008efe17a2308"
server_key_version: "encryptionkey@0"
I20260811 14:35:38.857430 7822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "91155c7391d1433ba68b5998872412c0"
format_stamp: "Formatted at 2026-08-11 14:35:38 on dist-test-slave-twwt"
server_key: "c43886253bfd9471038ba2d38634280f"
server_key_iv: "68e0def97dd11cbe5e9008efe17a2308"
server_key_version: "encryptionkey@0"
I20260811 14:35:38.858886 7822 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:38.859859 7848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:38.860090 7822 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:38.860226 7822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "91155c7391d1433ba68b5998872412c0"
format_stamp: "Formatted at 2026-08-11 14:35:38 on dist-test-slave-twwt"
server_key: "c43886253bfd9471038ba2d38634280f"
server_key_iv: "68e0def97dd11cbe5e9008efe17a2308"
server_key_version: "encryptionkey@0"
I20260811 14:35:38.860333 7822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:38.878357 7822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:38.880530 7822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:38.880712 7822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:38.880983 7822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:38.881341 7822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:38.881403 7822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:38.881453 7822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:38.881481 7822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:38.888192 7822 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:37549
I20260811 14:35:38.888252 8029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:37549 every 8 connection(s)
I20260811 14:35:38.888645 7822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:35:38.892064 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 7822
I20260811 14:35:38.892208 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:35:38.892413 2461 external_mini_cluster.cc:1428] Setting key ee12ac0f11d7be5b29a188f9ac1e0225
Aug 11 14:35:38 dist-test-slave-twwt krb5kdc[7433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458938, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:38.899753 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:38.889472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:36917 (local address 127.2.103.126:43905)
0811 14:35:38.889542 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:38.889547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:38.889801 (+ 254us) server_negotiation.cc:407] Connection header received
0811 14:35:38.890385 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:38.890386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:38.890396 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:38.890458 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:38.891541 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:38.892059 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:38.893400 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:38.893597 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:38.893674 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:38.896038 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:38.896039 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:38.896040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:38.896047 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:38.898184 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:38.898649 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:38.898650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:38.898651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:38.898721 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:38.898978 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:38.898979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:38.898980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:38.899147 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:38.899230 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:38.899516 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 14:35:38.899612 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 14:35:38.899776 8033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:38.889459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:43905 (local address 127.2.103.66:36917)
0811 14:35:38.889653 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:38.889700 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:38.890280 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:38.890523 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:38.890526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:38.891204 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:38.891435 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:38.891441 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:38.892173 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:38.892175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:38.893299 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:38.893302 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:38.893684 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:38.893685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:38.893819 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:38.894380 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:38.894389 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:38.895864 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:38.898291 (+ 2427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:38.898298 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:38.898300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:38.898548 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:38.898795 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:38.898796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:38.898796 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:38.898906 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:38.899227 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:38.899233 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:38.899370 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 14:35:38.899463 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 14:35:38.899594 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":108,"threads_started":1}
I20260811 14:35:38.900461 8030 heartbeater.cc:347] Connected to a master server at 127.2.103.126:43905
I20260811 14:35:38.900552 8030 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:38.900769 8030 heartbeater.cc:511] Master 127.2.103.126:43905 requested a full tablet report, sending...
I20260811 14:35:38.901232 7509 ts_manager.cc:194] Registered new tserver with Master: 91155c7391d1433ba68b5998872412c0 (127.2.103.66:37549)
I20260811 14:35:38.901624 7509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:36917
WARNING: no policy specified for kudu/127.2.103.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
I20260811 14:35:38.921957 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:43905
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:33653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:39.010232 8036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:39.010492 8036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:39.010536 8036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:39.011750 8036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:39.011834 8036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:39.011883 8036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:35:39.013439 8036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:33653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:43905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:39.013736 8036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:39.014004 8036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:39.019294 8045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:39.019307 8048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:39.019397 8051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:39.019711 8036 server_base.cc:1034] running on GCE node
I20260811 14:35:39.019934 8036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:39.020196 8036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:39.021368 8036 hybrid_clock.cc:648] HybridClock initialized: now 1786458939021352 us; error 36 us; skew 500 ppm
Aug 11 14:35:39 dist-test-slave-twwt krb5kdc[7433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458939, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:39.024091 8036 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:35:39.024610 8036 webserver.cc:460] Webserver started at http://127.2.103.67:41519/ using document root <none> and password file <none>
I20260811 14:35:39.024861 8036 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:39.024933 8036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:39.025085 8036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:39.026185 8036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "86ea36abb4474dccb7fe6a4cd31ab59c"
format_stamp: "Formatted at 2026-08-11 14:35:39 on dist-test-slave-twwt"
server_key: "e02774cd2571763eac5ee1f876452de3"
server_key_iv: "50dbb5703a26b1b35c8610c3ce9a71d5"
server_key_version: "encryptionkey@0"
I20260811 14:35:39.026577 8036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "86ea36abb4474dccb7fe6a4cd31ab59c"
format_stamp: "Formatted at 2026-08-11 14:35:39 on dist-test-slave-twwt"
server_key: "e02774cd2571763eac5ee1f876452de3"
server_key_iv: "50dbb5703a26b1b35c8610c3ce9a71d5"
server_key_version: "encryptionkey@0"
I20260811 14:35:39.028044 8036 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:39.028870 8062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:39.029073 8036 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:39.029183 8036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "86ea36abb4474dccb7fe6a4cd31ab59c"
format_stamp: "Formatted at 2026-08-11 14:35:39 on dist-test-slave-twwt"
server_key: "e02774cd2571763eac5ee1f876452de3"
server_key_iv: "50dbb5703a26b1b35c8610c3ce9a71d5"
server_key_version: "encryptionkey@0"
I20260811 14:35:39.029261 8036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:39.043737 8036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:39.046406 8036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:39.046535 8036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:39.046856 8036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:39.047149 8036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:39.047180 8036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:39.047204 8036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:39.047224 8036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:39.053248 8036 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:34403
I20260811 14:35:39.053347 8246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:34403 every 8 connection(s)
I20260811 14:35:39.053696 8036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:35:39.058336 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 8036
I20260811 14:35:39.058444 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:35:39.058691 2461 external_mini_cluster.cc:1428] Setting key ca0d5ee70f5b5c148674cbd25c6f07c9
Aug 11 14:35:39 dist-test-slave-twwt krb5kdc[7433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458939, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:39.064067 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:39.054519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:49563 (local address 127.2.103.126:43905)
0811 14:35:39.054587 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:39.054590 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:39.054821 (+ 231us) server_negotiation.cc:407] Connection header received
0811 14:35:39.055429 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:39.055430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:39.055443 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:39.055503 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:39.056593 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:39.057111 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:39.058134 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:39.058446 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:39.058518 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:39.061085 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:39.061087 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:39.061087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:39.061095 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:39.062736 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:39.063177 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:39.063178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:39.063179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:39.063239 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:39.063504 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:39.063505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:39.063505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:39.063619 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:39.063674 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:39.063922 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 14:35:39.064001 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 14:35:39.064311 8250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:39.054507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:43905 (local address 127.2.103.67:49563)
0811 14:35:39.054695 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:39.054733 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:39.055298 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:39.055587 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:39.055591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:39.056259 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:39.056473 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:39.056479 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:39.057228 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:39.057229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:39.058037 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:39.058038 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:39.058573 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:39.058574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:39.058792 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:39.059429 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:39.059438 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:39.060938 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:39.062821 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:39.062828 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:39.062830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:39.063075 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:39.063315 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:39.063315 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:39.063316 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:39.063438 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:39.063680 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:39.063684 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:39.063807 (+ 123us) client_negotiation.cc:769] Sending connection context
0811 14:35:39.063893 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 14:35:39.064022 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":75,"threads_started":1}
I20260811 14:35:39.064920 8247 heartbeater.cc:347] Connected to a master server at 127.2.103.126:43905
I20260811 14:35:39.065042 8247 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:39.065234 8247 heartbeater.cc:511] Master 127.2.103.126:43905 requested a full tablet report, sending...
I20260811 14:35:39.065639 7509 ts_manager.cc:194] Registered new tserver with Master: 86ea36abb4474dccb7fe6a4cd31ab59c (127.2.103.67:34403)
I20260811 14:35:39.065999 7509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:49563
I20260811 14:35:39.070591 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 14:35:39 dist-test-slave-twwt krb5kdc[7433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458939, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 14:35:39 dist-test-slave-twwt krb5kdc[7433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458939, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:39.097070 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:39.089165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34478 (local address 127.2.103.126:43905)
0811 14:35:39.089244 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:39.089246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:39.089501 (+ 255us) server_negotiation.cc:407] Connection header received
0811 14:35:39.089568 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:39.089569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:39.089580 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:39.089635 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:39.090261 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:39.090850 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:39.091604 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:39.091791 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:39.091877 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:39.094307 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:39.094309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:39.094310 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:39.094317 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:39.095801 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:39.096233 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:39.096234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:39.096235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:39.096320 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:39.096581 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:39.096582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:39.096582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:39.096739 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:39.096821 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:39.096936 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 14:35:39.097000 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 14:35:39.097419 7509 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34478
I20260811 14:35:39.097877 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 7602
I20260811 14:35:39.113648 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 7822
I20260811 14:35:39.129094 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 8036
I20260811 14:35:39.150303 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 7451
2026-08-11T14:35:39Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2205 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:39 dist-test-slave-twwt krb5kdc[8267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:39 dist-test-slave-twwt krb5kdc[8267](info): set up 2 sockets
Aug 11 14:35:39 dist-test-slave-twwt krb5kdc[8267](info): commencing operation
krb5kdc: starting...
W20260811 14:35:40.443557 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:35:40 dist-test-slave-twwt krb5kdc[8267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:35:40.577962 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:35539
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:42451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:35539
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:40.664659 8292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:40.664939 8292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:40.664984 8292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:40.666232 8292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:40.666311 8292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:40.666335 8292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:40.666373 8292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:40.666404 8292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:40.668090 8292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:42451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:35539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:35539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:40.668392 8292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:40.668646 8292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:40.674331 8302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:40.674417 8306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:40.674336 8301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:40.674535 8292 server_base.cc:1034] running on GCE node
I20260811 14:35:40.674873 8292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:40.675256 8292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:40.676422 8292 hybrid_clock.cc:648] HybridClock initialized: now 1786458940676400 us; error 49 us; skew 500 ppm
Aug 11 14:35:40 dist-test-slave-twwt krb5kdc[8267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458940, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:40.685402 8292 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:35:40.685992 8292 webserver.cc:460] Webserver started at http://127.2.103.126:44335/ using document root <none> and password file <none>
I20260811 14:35:40.686242 8292 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:40.686316 8292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:40.686479 8292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:40.687772 8292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "c61d66b0010344a696645182de64e478"
format_stamp: "Formatted at 2026-08-11 14:35:40 on dist-test-slave-twwt"
server_key: "3e3dbfe24889161d6486386f848eba46"
server_key_iv: "8cf0d874713293917d78668659213f50"
server_key_version: "encryptionkey@0"
I20260811 14:35:40.688192 8292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "c61d66b0010344a696645182de64e478"
format_stamp: "Formatted at 2026-08-11 14:35:40 on dist-test-slave-twwt"
server_key: "3e3dbfe24889161d6486386f848eba46"
server_key_iv: "8cf0d874713293917d78668659213f50"
server_key_version: "encryptionkey@0"
I20260811 14:35:40.689692 8292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:35:40.690544 8317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:40.690763 8292 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:35:40.690909 8292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "c61d66b0010344a696645182de64e478"
format_stamp: "Formatted at 2026-08-11 14:35:40 on dist-test-slave-twwt"
server_key: "3e3dbfe24889161d6486386f848eba46"
server_key_iv: "8cf0d874713293917d78668659213f50"
server_key_version: "encryptionkey@0"
I20260811 14:35:40.690996 8292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:40.715407 8292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:40.717756 8292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:40.717991 8292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:40.722676 8292 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:35539
I20260811 14:35:40.722765 8407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:35539 every 8 connection(s)
I20260811 14:35:40.723400 8292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:40.724187 8408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:40.724721 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 8292
I20260811 14:35:40.724901 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:35:40.725090 2461 external_mini_cluster.cc:1428] Setting key 141795c862a33c374eac1245aea4906c
I20260811 14:35:40.727325 8408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478: Bootstrap starting.
I20260811 14:35:40.727989 8408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:40.728288 8408 log.cc:824] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:40.729133 8408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478: No bootstrap required, opened a new log
I20260811 14:35:40.731817 8408 raft_consensus.cc:348] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61d66b0010344a696645182de64e478" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35539 } }
I20260811 14:35:40.731972 8408 raft_consensus.cc:374] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:40.731997 8408 raft_consensus.cc:729] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c61d66b0010344a696645182de64e478, State: Initialized, Role: FOLLOWER
I20260811 14:35:40.732134 8408 consensus_queue.cc:260] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61d66b0010344a696645182de64e478" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35539 } }
I20260811 14:35:40.732220 8408 raft_consensus.cc:388] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:40.732244 8408 raft_consensus.cc:482] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:40.732313 8408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478 [term 0 FOLLOWER]: Advancing to term 1
Aug 11 14:35:40 dist-test-slave-twwt krb5kdc[8267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:40.732980 8408 raft_consensus.cc:504] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61d66b0010344a696645182de64e478" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35539 } }
I20260811 14:35:40.733115 8408 leader_election.cc:302] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c61d66b0010344a696645182de64e478; no voters:
I20260811 14:35:40.733301 8408 leader_election.cc:288] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:40.733366 8414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:40.733528 8414 raft_consensus.cc:686] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478 [term 1 LEADER]: Becoming Leader. State: Replica: c61d66b0010344a696645182de64e478, State: Running, Role: LEADER
I20260811 14:35:40.733624 8408 sys_catalog.cc:564] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:40.733644 8414 consensus_queue.cc:237] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61d66b0010344a696645182de64e478" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35539 } }
I20260811 14:35:40.734412 8416 sys_catalog.cc:455] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c61d66b0010344a696645182de64e478. Latest consensus state: current_term: 1 leader_uuid: "c61d66b0010344a696645182de64e478" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61d66b0010344a696645182de64e478" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35539 } } }
I20260811 14:35:40.734513 8416 sys_catalog.cc:458] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:40.735311 8422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:40.735957 8422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:40.734692 8418 sys_catalog.cc:455] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c61d66b0010344a696645182de64e478" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61d66b0010344a696645182de64e478" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35539 } } }
I20260811 14:35:40.736402 8418 sys_catalog.cc:458] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:40.737643 8411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:40.725696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40758 (local address 127.2.103.126:35539)
0811 14:35:40.725873 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:40.725881 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:40.725908 (+ 27us) server_negotiation.cc:407] Connection header received
0811 14:35:40.726323 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:40.726329 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:40.726377 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:40.726491 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:40.727375 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:40.728438 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:40.729443 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:40.729875 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:40.729961 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:40.732993 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:40.732998 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:40.732998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:40.733013 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:40.735756 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:40.736197 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:40.736201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:40.736201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:40.736311 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:40.736565 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:40.736566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:40.736567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:40.736896 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:40.737037 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:40.737289 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 14:35:40.737435 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
I20260811 14:35:40.737907 8422 catalog_manager.cc:1347] Generated new cluster ID: 6cb47eea2c8b4208be0ef7dcb1b50a63
I20260811 14:35:40.737962 8422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:35:40.760052 8422 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:40.760649 8422 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:40.767493 8422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c61d66b0010344a696645182de64e478: Generated new TSK 0
I20260811 14:35:40.767767 8422 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
I20260811 14:35:40.809705 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:35539
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:42451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:40.920598 8454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:40.920815 8454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:40.920841 8454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:40.922024 8454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:40.922066 8454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:40.922093 8454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:35:40.923671 8454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:42451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:35539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:40.923931 8454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:40.924178 8454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:40.929427 8466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:40.929461 8468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:40.929680 8465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:40.929651 8454 server_base.cc:1034] running on GCE node
I20260811 14:35:40.929936 8454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:40.930234 8454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:40.931432 8454 hybrid_clock.cc:648] HybridClock initialized: now 1786458940931416 us; error 44 us; skew 500 ppm
Aug 11 14:35:40 dist-test-slave-twwt krb5kdc[8267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458940, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:40.934430 8454 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:35:40.935117 8454 webserver.cc:460] Webserver started at http://127.2.103.65:37193/ using document root <none> and password file <none>
I20260811 14:35:40.935386 8454 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:40.935459 8454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:40.935639 8454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:40.937037 8454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "9197a2523132456b88d49827dec25aaf"
format_stamp: "Formatted at 2026-08-11 14:35:40 on dist-test-slave-twwt"
server_key: "4fa494286add998a9e76881d1da7eea8"
server_key_iv: "8ef7880250e0232dbf44a8bf3b184cde"
server_key_version: "encryptionkey@0"
I20260811 14:35:40.937476 8454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "9197a2523132456b88d49827dec25aaf"
format_stamp: "Formatted at 2026-08-11 14:35:40 on dist-test-slave-twwt"
server_key: "4fa494286add998a9e76881d1da7eea8"
server_key_iv: "8ef7880250e0232dbf44a8bf3b184cde"
server_key_version: "encryptionkey@0"
I20260811 14:35:40.939105 8454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260811 14:35:40.939967 8477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:40.940193 8454 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:40.940315 8454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "9197a2523132456b88d49827dec25aaf"
format_stamp: "Formatted at 2026-08-11 14:35:40 on dist-test-slave-twwt"
server_key: "4fa494286add998a9e76881d1da7eea8"
server_key_iv: "8ef7880250e0232dbf44a8bf3b184cde"
server_key_version: "encryptionkey@0"
I20260811 14:35:40.940390 8454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:40.957376 8454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:40.959458 8454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:40.959614 8454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:40.959885 8454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:40.960215 8454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:40.960278 8454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:40.960320 8454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:40.960350 8454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:40.966387 8454 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:46863
I20260811 14:35:40.966449 8658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:46863 every 8 connection(s)
I20260811 14:35:40.966876 8454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:35:40 dist-test-slave-twwt krb5kdc[8267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458940, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:40.976780 8411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:40.967694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:49935 (local address 127.2.103.126:35539)
0811 14:35:40.967744 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:40.967747 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:40.967997 (+ 250us) server_negotiation.cc:407] Connection header received
0811 14:35:40.969055 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:40.969056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:40.969071 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:40.969127 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:40.969808 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:40.970324 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:40.971367 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:40.971590 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:40.971682 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:40.973992 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:40.973994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:40.973994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:40.974001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:40.975320 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:40.975754 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:40.975755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:40.975755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:40.975815 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:40.976140 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:40.976141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:40.976141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:40.976260 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:40.976321 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:40.976639 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 14:35:40.976712 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:35:40.977157 8666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:40.967663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:35539 (local address 127.2.103.65:49935)
0811 14:35:40.967861 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:40.967904 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:40.968929 (+ 1025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:40.969208 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:40.969212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:40.969500 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:40.969699 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:40.969704 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:40.970435 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:40.970436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:40.971268 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:40.971271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:40.971695 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:40.971696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:40.971826 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:40.972399 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:40.972409 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:40.973897 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:40.975425 (+ 1528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:40.975433 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:40.975435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:40.975673 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:40.975925 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:40.975926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:40.975927 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:40.976049 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:40.976365 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:40.976372 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:40.976512 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 14:35:40.976613 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 14:35:40.976753 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":92,"threads_started":1}
I20260811 14:35:40.977221 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 8454
I20260811 14:35:40.977345 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:35:40.977547 2461 external_mini_cluster.cc:1428] Setting key 658ebe0240f7b3a0b45ca237378dc482
I20260811 14:35:40.977835 8659 heartbeater.cc:347] Connected to a master server at 127.2.103.126:35539
I20260811 14:35:40.977955 8659 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:40.978204 8659 heartbeater.cc:511] Master 127.2.103.126:35539 requested a full tablet report, sending...
I20260811 14:35:40.978852 8342 ts_manager.cc:194] Registered new tserver with Master: 9197a2523132456b88d49827dec25aaf (127.2.103.65:46863)
I20260811 14:35:40.979465 8342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:49935
WARNING: no policy specified for kudu/127.2.103.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
I20260811 14:35:41.006186 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:35539
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:42451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:41.093050 8672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:41.093317 8672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:41.093344 8672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:41.094516 8672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:41.094558 8672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:41.094584 8672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:35:41.096138 8672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:42451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:35539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:41.096341 8672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:41.096526 8672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:41.101753 8684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:41.101809 8680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:41.101899 8672 server_base.cc:1034] running on GCE node
W20260811 14:35:41.101753 8679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:41.102243 8672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:41.102512 8672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:41.103673 8672 hybrid_clock.cc:648] HybridClock initialized: now 1786458941103666 us; error 40 us; skew 500 ppm
Aug 11 14:35:41 dist-test-slave-twwt krb5kdc[8267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458941, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:41.106508 8672 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:35:41.107112 8672 webserver.cc:460] Webserver started at http://127.2.103.66:39855/ using document root <none> and password file <none>
I20260811 14:35:41.107405 8672 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:41.107484 8672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:41.107657 8672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:41.108888 8672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "4b6b4af939f843ac8bbf6ff4e835311a"
format_stamp: "Formatted at 2026-08-11 14:35:41 on dist-test-slave-twwt"
server_key: "ec0a9250873aa3bf5f6eb67c8c720513"
server_key_iv: "d52f606e9eda2b62049a76ff25019fe7"
server_key_version: "encryptionkey@0"
I20260811 14:35:41.109321 8672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "4b6b4af939f843ac8bbf6ff4e835311a"
format_stamp: "Formatted at 2026-08-11 14:35:41 on dist-test-slave-twwt"
server_key: "ec0a9250873aa3bf5f6eb67c8c720513"
server_key_iv: "d52f606e9eda2b62049a76ff25019fe7"
server_key_version: "encryptionkey@0"
I20260811 14:35:41.110911 8672 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:41.111703 8694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:41.111891 8672 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:41.112001 8672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "4b6b4af939f843ac8bbf6ff4e835311a"
format_stamp: "Formatted at 2026-08-11 14:35:41 on dist-test-slave-twwt"
server_key: "ec0a9250873aa3bf5f6eb67c8c720513"
server_key_iv: "d52f606e9eda2b62049a76ff25019fe7"
server_key_version: "encryptionkey@0"
I20260811 14:35:41.112100 8672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:41.123349 8672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:41.125788 8672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:41.125963 8672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:41.126207 8672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:41.126535 8672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:41.126591 8672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:41.126648 8672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:41.126699 8672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:41.133090 8672 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:35119
I20260811 14:35:41.133183 8892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:35119 every 8 connection(s)
I20260811 14:35:41.133527 8672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
Aug 11 14:35:41 dist-test-slave-twwt krb5kdc[8267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458941, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:41.142517 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 8672
I20260811 14:35:41.142779 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:35:41.142988 2461 external_mini_cluster.cc:1428] Setting key c620b87aad10899575449c56a6582f39
I20260811 14:35:41.143420 8411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:41.134304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:41817 (local address 127.2.103.126:35539)
0811 14:35:41.134351 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:41.134353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:41.134594 (+ 241us) server_negotiation.cc:407] Connection header received
0811 14:35:41.135724 (+ 1130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:41.135726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:41.135739 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:41.135794 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:41.136436 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:41.136945 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:41.137945 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:41.138164 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:41.138242 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:41.140497 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:41.140498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:41.140499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:41.140507 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:41.141814 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:41.142252 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:41.142253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:41.142253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:41.142334 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:41.142628 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:41.142629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:41.142629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:41.142788 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:41.142856 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:41.143175 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 14:35:41.143316 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:35:41.143625 8898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:41.134296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:35539 (local address 127.2.103.66:41817)
0811 14:35:41.134474 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:41.134509 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:41.135594 (+ 1085us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:41.135870 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:41.135874 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:41.136136 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:41.136329 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:41.136333 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:41.137046 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:41.137047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:41.137845 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:41.137847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:41.138248 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:41.138250 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:41.138389 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:41.139011 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:41.139020 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:41.140394 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:41.141906 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:41.141914 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:41.141916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:41.142161 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:41.142417 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:41.142418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:41.142418 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:41.142530 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:41.142847 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:41.142854 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:41.143004 (+ 150us) client_negotiation.cc:769] Sending connection context
0811 14:35:41.143142 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 14:35:41.143353 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":129,"threads_started":1}
I20260811 14:35:41.145823 8894 heartbeater.cc:347] Connected to a master server at 127.2.103.126:35539
I20260811 14:35:41.145947 8894 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:41.146143 8894 heartbeater.cc:511] Master 127.2.103.126:35539 requested a full tablet report, sending...
I20260811 14:35:41.146564 8342 ts_manager.cc:194] Registered new tserver with Master: 4b6b4af939f843ac8bbf6ff4e835311a (127.2.103.66:35119)
I20260811 14:35:41.146981 8342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:41817
WARNING: no policy specified for kudu/127.2.103.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
I20260811 14:35:41.174052 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:35539
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:42451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:41.259866 8902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:41.260100 8902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:41.260138 8902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:41.261304 8902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:41.261374 8902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:41.261430 8902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:35:41.263026 8902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:42451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:35539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:41.263302 8902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:41.263554 8902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:41.268939 8911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:41.268976 8909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:41.269186 8902 server_base.cc:1034] running on GCE node
W20260811 14:35:41.269265 8907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:41.269556 8902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:41.269826 8902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:41.271091 8902 hybrid_clock.cc:648] HybridClock initialized: now 1786458941271065 us; error 28 us; skew 500 ppm
Aug 11 14:35:41 dist-test-slave-twwt krb5kdc[8267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458941, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:41.274219 8902 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:35:41.274806 8902 webserver.cc:460] Webserver started at http://127.2.103.67:41957/ using document root <none> and password file <none>
I20260811 14:35:41.275084 8902 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:41.275163 8902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:41.275307 8902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:41.276441 8902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "bdda7a2a0e8f484d99310584a47b1ec0"
format_stamp: "Formatted at 2026-08-11 14:35:41 on dist-test-slave-twwt"
server_key: "49d4edf4af9d3af51e8d4580386d84f3"
server_key_iv: "6832fdc2b7b1da8f960ef653ebbbeac6"
server_key_version: "encryptionkey@0"
I20260811 14:35:41.276850 8902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "bdda7a2a0e8f484d99310584a47b1ec0"
format_stamp: "Formatted at 2026-08-11 14:35:41 on dist-test-slave-twwt"
server_key: "49d4edf4af9d3af51e8d4580386d84f3"
server_key_iv: "6832fdc2b7b1da8f960ef653ebbbeac6"
server_key_version: "encryptionkey@0"
I20260811 14:35:41.278278 8902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:35:41.279162 8920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:41.279366 8902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:35:41.279464 8902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "bdda7a2a0e8f484d99310584a47b1ec0"
format_stamp: "Formatted at 2026-08-11 14:35:41 on dist-test-slave-twwt"
server_key: "49d4edf4af9d3af51e8d4580386d84f3"
server_key_iv: "6832fdc2b7b1da8f960ef653ebbbeac6"
server_key_version: "encryptionkey@0"
I20260811 14:35:41.279552 8902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:41.303800 8902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:41.306334 8902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:41.306509 8902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:41.306808 8902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:41.307142 8902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:41.307199 8902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:41.307236 8902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:41.307266 8902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:41.313594 8902 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:34161
I20260811 14:35:41.313645 9117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:34161 every 8 connection(s)
I20260811 14:35:41.313984 8902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
Aug 11 14:35:41 dist-test-slave-twwt krb5kdc[8267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458941, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:41.320580 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 8902
I20260811 14:35:41.320693 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:35:41.320878 2461 external_mini_cluster.cc:1428] Setting key 63fec7de85b710df34a76faa1247aed9
I20260811 14:35:41.324142 8411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:41.314781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:52639 (local address 127.2.103.126:35539)
0811 14:35:41.314850 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:41.314852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:41.315101 (+ 249us) server_negotiation.cc:407] Connection header received
0811 14:35:41.316243 (+ 1142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:41.316245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:41.316258 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:41.316301 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:41.316964 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:41.317471 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:41.318493 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:41.318764 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:41.318841 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:41.321139 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:41.321141 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:41.321141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:41.321150 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:41.322661 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:41.323102 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:41.323104 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:41.323104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:41.323167 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:41.323437 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:41.323438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:41.323438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:41.323592 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:41.323673 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:41.323966 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 14:35:41.324049 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 14:35:41.324330 9122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:41.314763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:35539 (local address 127.2.103.67:52639)
0811 14:35:41.314961 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:41.315013 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:41.316112 (+ 1099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:41.316388 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:41.316392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:41.316654 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:41.316854 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:41.316858 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:41.317574 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:41.317575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:41.318400 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:41.318402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:41.318863 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:41.318865 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:41.319007 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:41.319469 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:41.319478 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:41.321016 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:41.322760 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:41.322768 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:41.322770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:41.323007 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:41.323242 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:41.323243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:41.323243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:41.323359 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:41.323675 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:41.323683 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:41.323824 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 14:35:41.323932 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 14:35:41.324093 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":91,"threads_started":1}
I20260811 14:35:41.325026 9118 heartbeater.cc:347] Connected to a master server at 127.2.103.126:35539
I20260811 14:35:41.325114 9118 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:41.325314 9118 heartbeater.cc:511] Master 127.2.103.126:35539 requested a full tablet report, sending...
I20260811 14:35:41.325793 8342 ts_manager.cc:194] Registered new tserver with Master: bdda7a2a0e8f484d99310584a47b1ec0 (127.2.103.67:34161)
I20260811 14:35:41.326205 8342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:52639
I20260811 14:35:41.332686 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 14:35:41 dist-test-slave-twwt krb5kdc[8267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 14:35:41 dist-test-slave-twwt krb5kdc[8267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.65@KRBTEST.COM
I20260811 14:35:41.359503 9132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:41.351079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37644 (local address 127.2.103.65:46863)
0811 14:35:41.351267 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:41.351272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:41.351284 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:35:41.351317 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:41.351319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:41.351335 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:41.351486 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:41.352074 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:41.352811 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:41.353548 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:41.353976 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:41.354070 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:41.356582 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:41.356591 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:41.356591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:41.356598 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:41.358310 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:41.358704 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:41.358707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:41.358707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:41.358776 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:41.358990 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:41.358991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:41.358991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:41.359132 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:41.359237 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:41.359327 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 14:35:41.359389 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":97,"threads_started":1}
Aug 11 14:35:41 dist-test-slave-twwt krb5kdc[8267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:41.368120 8411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:41.360954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40766 (local address 127.2.103.126:35539)
0811 14:35:41.360998 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:41.361000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:41.361010 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:35:41.361060 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:41.361061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:41.361070 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:41.361121 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:41.361639 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:41.362122 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:41.362904 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:41.363091 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:41.363150 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:41.365525 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:41.365527 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:41.365527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:41.365535 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:41.366908 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:41.367239 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:41.367240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:41.367240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:41.367296 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:41.367537 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:41.367538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:41.367538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:41.367693 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:41.367773 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:41.367933 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 14:35:41.368003 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260811 14:35:41.368398 8342 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40766
I20260811 14:35:41.368760 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 8454
I20260811 14:35:41.387679 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 8672
I20260811 14:35:41.406430 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 8902
I20260811 14:35:41.425848 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 8292
2026-08-11T14:35:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2275 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:41 dist-test-slave-twwt krb5kdc[9141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:41 dist-test-slave-twwt krb5kdc[9141](info): set up 2 sockets
Aug 11 14:35:41 dist-test-slave-twwt krb5kdc[9141](info): commencing operation
krb5kdc: starting...
W20260811 14:35:42.697701 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:35:42 dist-test-slave-twwt krb5kdc[9141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:35:42.831414 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:46181
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:35351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:46181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:42.918541 9164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:42.918790 9164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:42.918814 9164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:42.920028 9164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:42.920074 9164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:42.920089 9164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:42.920101 9164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:42.920153 9164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:42.921723 9164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:35351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:46181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:46181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:42.922000 9164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:42.922246 9164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:42.927285 9170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:42.927302 9173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:42.927289 9169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:42.927505 9164 server_base.cc:1034] running on GCE node
I20260811 14:35:42.927896 9164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:42.928208 9164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:42.929374 9164 hybrid_clock.cc:648] HybridClock initialized: now 1786458942929350 us; error 55 us; skew 500 ppm
Aug 11 14:35:42 dist-test-slave-twwt krb5kdc[9141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458942, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:42.932135 9164 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:35:42.932515 9164 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:42.932614 9164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:42.932775 9164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:42.933949 9164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "5430e27dcbe044beaee9c001b2289555"
format_stamp: "Formatted at 2026-08-11 14:35:42 on dist-test-slave-twwt"
server_key: "03568ccdd9db01e48b437655d78f52f0"
server_key_iv: "69df41b68a2284748bf9eacbe7b6104b"
server_key_version: "encryptionkey@0"
I20260811 14:35:42.934343 9164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "5430e27dcbe044beaee9c001b2289555"
format_stamp: "Formatted at 2026-08-11 14:35:42 on dist-test-slave-twwt"
server_key: "03568ccdd9db01e48b437655d78f52f0"
server_key_iv: "69df41b68a2284748bf9eacbe7b6104b"
server_key_version: "encryptionkey@0"
I20260811 14:35:42.935881 9164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:42.936717 9186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:42.936946 9164 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:42.937058 9164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "5430e27dcbe044beaee9c001b2289555"
format_stamp: "Formatted at 2026-08-11 14:35:42 on dist-test-slave-twwt"
server_key: "03568ccdd9db01e48b437655d78f52f0"
server_key_iv: "69df41b68a2284748bf9eacbe7b6104b"
server_key_version: "encryptionkey@0"
I20260811 14:35:42.937146 9164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:42.966313 9164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:42.968927 9164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:42.969056 9164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:42.973318 9164 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:46181
I20260811 14:35:42.973450 9266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:46181 every 8 connection(s)
I20260811 14:35:42.973721 9164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:42.974968 9269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:42.977519 9269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555: Bootstrap starting.
I20260811 14:35:42.977977 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 9164
I20260811 14:35:42.978121 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:35:42.978305 9269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:42.978336 2461 external_mini_cluster.cc:1428] Setting key 297ca6e7f3f12bcea1695c7ffda578da
I20260811 14:35:42.978708 9269 log.cc:824] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:42.979516 9269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555: No bootstrap required, opened a new log
I20260811 14:35:42.981931 9269 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5430e27dcbe044beaee9c001b2289555" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46181 } }
I20260811 14:35:42.982137 9269 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:42.982182 9269 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5430e27dcbe044beaee9c001b2289555, State: Initialized, Role: FOLLOWER
I20260811 14:35:42.982324 9269 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5430e27dcbe044beaee9c001b2289555" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46181 } }
I20260811 14:35:42.982406 9269 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:42.982436 9269 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:42.982481 9269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:42.983217 9269 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5430e27dcbe044beaee9c001b2289555" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46181 } }
I20260811 14:35:42.983348 9269 leader_election.cc:302] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5430e27dcbe044beaee9c001b2289555; no voters:
I20260811 14:35:42.983541 9269 leader_election.cc:288] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:42.983585 9275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:42.983772 9275 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555 [term 1 LEADER]: Becoming Leader. State: Replica: 5430e27dcbe044beaee9c001b2289555, State: Running, Role: LEADER
I20260811 14:35:42.983848 9269 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:42.983918 9275 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5430e27dcbe044beaee9c001b2289555" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46181 } }
I20260811 14:35:42.985486 9276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5430e27dcbe044beaee9c001b2289555" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5430e27dcbe044beaee9c001b2289555" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46181 } } }
I20260811 14:35:42.985579 9276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:42.985827 9277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5430e27dcbe044beaee9c001b2289555. Latest consensus state: current_term: 1 leader_uuid: "5430e27dcbe044beaee9c001b2289555" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5430e27dcbe044beaee9c001b2289555" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46181 } } }
Aug 11 14:35:42 dist-test-slave-twwt krb5kdc[9141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
W20260811 14:35:42.985913 9303 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260811 14:35:42.985953 9277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555 [sys.catalog]: This master's current role is: LEADER
W20260811 14:35:42.985962 9303 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:35:42.986037 9304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:42.986259 9304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:42.988058 9304 catalog_manager.cc:1347] Generated new cluster ID: 2190f6de6849499cbc076cbcd1adde14
I20260811 14:35:42.988122 9304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:35:42.990711 9273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:42.978984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35218 (local address 127.2.103.126:46181)
0811 14:35:42.979231 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:42.979242 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:42.979280 (+ 38us) server_negotiation.cc:407] Connection header received
0811 14:35:42.979787 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:42.979795 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:42.979849 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:42.979982 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:42.980826 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:42.981726 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:42.982848 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:42.983278 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:42.983421 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:42.986514 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:42.986519 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:42.986519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:42.986532 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:42.988914 (+ 2382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:42.989397 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:42.989400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:42.989401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:42.989508 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:42.989731 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:42.989732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:42.989733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:42.990028 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:42.990177 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:42.990329 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 14:35:42.990472 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":128,"threads_started":1}
I20260811 14:35:43.007354 9304 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:43.007972 9304 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:43.018537 9304 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555: Generated new TSK 0
I20260811 14:35:43.018834 9304 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
I20260811 14:35:43.059144 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:46181
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:35351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:43.146095 9317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:43.146323 9317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:43.146349 9317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:43.147605 9317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:43.147655 9317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:43.147683 9317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:35:43.149314 9317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:35351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:46181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:43.149612 9317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:43.149871 9317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:43.156771 9325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:43.156890 9317 server_base.cc:1034] running on GCE node
W20260811 14:35:43.156785 9324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:43.156771 9327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:43.157274 9317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:43.157557 9317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:43.158664 9317 hybrid_clock.cc:648] HybridClock initialized: now 1786458943158652 us; error 29 us; skew 500 ppm
Aug 11 14:35:43 dist-test-slave-twwt krb5kdc[9141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458943, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:43.161306 9317 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:35:43.161743 9317 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:43.161818 9317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:43.161978 9317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:43.163170 9317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "f66422f7706e4f6c8628571f196281cc"
format_stamp: "Formatted at 2026-08-11 14:35:43 on dist-test-slave-twwt"
server_key: "fbbe545f150e96d938def751f54962e6"
server_key_iv: "1d81d1618b0139a0e1baf4fed50a7828"
server_key_version: "encryptionkey@0"
I20260811 14:35:43.163552 9317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "f66422f7706e4f6c8628571f196281cc"
format_stamp: "Formatted at 2026-08-11 14:35:43 on dist-test-slave-twwt"
server_key: "fbbe545f150e96d938def751f54962e6"
server_key_iv: "1d81d1618b0139a0e1baf4fed50a7828"
server_key_version: "encryptionkey@0"
I20260811 14:35:43.164921 9317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:35:43.165731 9335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:43.165937 9317 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:43.166046 9317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "f66422f7706e4f6c8628571f196281cc"
format_stamp: "Formatted at 2026-08-11 14:35:43 on dist-test-slave-twwt"
server_key: "fbbe545f150e96d938def751f54962e6"
server_key_iv: "1d81d1618b0139a0e1baf4fed50a7828"
server_key_version: "encryptionkey@0"
I20260811 14:35:43.166126 9317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:43.188596 9317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:43.190725 9317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:43.190910 9317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:43.191164 9317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:43.191514 9317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:43.191568 9317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:43.191605 9317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:43.191633 9317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:43.197868 9317 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:44705
I20260811 14:35:43.197914 9515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:44705 every 8 connection(s)
I20260811 14:35:43.198199 9317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:35:43 dist-test-slave-twwt krb5kdc[9141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458943, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:43.206106 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 9317
I20260811 14:35:43.206243 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:35:43.206478 2461 external_mini_cluster.cc:1428] Setting key d1947e753f24bcf312f4dd7bdf6348cc
I20260811 14:35:43.207965 9273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:43.198979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:59301 (local address 127.2.103.126:46181)
0811 14:35:43.199069 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:43.199072 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:43.199236 (+ 164us) server_negotiation.cc:407] Connection header received
0811 14:35:43.199729 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:43.199730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:43.199740 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:43.199795 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:43.200752 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.201271 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.202311 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.202513 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.202571 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:43.204836 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:43.204837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:43.204838 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:43.204844 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:43.206390 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:43.206926 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:43.206928 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:43.206928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:43.206992 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:43.207272 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:43.207273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:43.207273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:43.207415 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:43.207471 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:43.207807 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 14:35:43.207898 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 14:35:43.208266 9519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:43.198929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46181 (local address 127.2.103.65:59301)
0811 14:35:43.199103 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:43.199142 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:43.199638 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:43.199857 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:43.199860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:43.200136 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:43.200665 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:43.200670 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.201366 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.201367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:43.202219 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:43.202222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.202591 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.202592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:43.202760 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:43.203254 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:43.203264 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:43.204739 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:43.206466 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:43.206473 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:43.206475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:43.206830 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:43.207056 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:43.207057 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:43.207057 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:43.207204 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:43.207482 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:43.207489 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:43.207667 (+ 178us) client_negotiation.cc:769] Sending connection context
0811 14:35:43.207788 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 14:35:43.207978 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
I20260811 14:35:43.208829 9517 heartbeater.cc:347] Connected to a master server at 127.2.103.126:46181
I20260811 14:35:43.208941 9517 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:43.209121 9517 heartbeater.cc:511] Master 127.2.103.126:46181 requested a full tablet report, sending...
I20260811 14:35:43.209738 9207 ts_manager.cc:194] Registered new tserver with Master: f66422f7706e4f6c8628571f196281cc (127.2.103.65:44705)
I20260811 14:35:43.210702 9207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:59301
WARNING: no policy specified for kudu/127.2.103.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
I20260811 14:35:43.235693 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:46181
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:35351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:43.323115 9523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:43.323329 9523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:43.323355 9523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:43.324499 9523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:43.324535 9523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:43.324563 9523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:35:43.326073 9523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:35351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:46181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:43.326289 9523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:43.326488 9523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:43.331701 9539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:43.331703 9534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:43.331735 9523 server_base.cc:1034] running on GCE node
W20260811 14:35:43.331703 9536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:43.332162 9523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:43.332417 9523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:43.333577 9523 hybrid_clock.cc:648] HybridClock initialized: now 1786458943333572 us; error 33 us; skew 500 ppm
Aug 11 14:35:43 dist-test-slave-twwt krb5kdc[9141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458943, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:43.336223 9523 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:35:43.336629 9523 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:43.336702 9523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:43.336849 9523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:43.337952 9523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "695c1771631a441ba142e473f2d7dec1"
format_stamp: "Formatted at 2026-08-11 14:35:43 on dist-test-slave-twwt"
server_key: "196c273a9a20e6360d86b1432c5ce0df"
server_key_iv: "de373a4db024b5c1525487c32e6f9563"
server_key_version: "encryptionkey@0"
I20260811 14:35:43.338325 9523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "695c1771631a441ba142e473f2d7dec1"
format_stamp: "Formatted at 2026-08-11 14:35:43 on dist-test-slave-twwt"
server_key: "196c273a9a20e6360d86b1432c5ce0df"
server_key_iv: "de373a4db024b5c1525487c32e6f9563"
server_key_version: "encryptionkey@0"
I20260811 14:35:43.339814 9523 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260811 14:35:43.340610 9551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:43.340801 9523 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:43.340909 9523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "695c1771631a441ba142e473f2d7dec1"
format_stamp: "Formatted at 2026-08-11 14:35:43 on dist-test-slave-twwt"
server_key: "196c273a9a20e6360d86b1432c5ce0df"
server_key_iv: "de373a4db024b5c1525487c32e6f9563"
server_key_version: "encryptionkey@0"
I20260811 14:35:43.340993 9523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:43.352078 9523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:43.354440 9523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:43.354712 9523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:43.354948 9523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:43.355221 9523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:43.355253 9523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:43.355278 9523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:43.355290 9523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:43.361227 9523 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:45945
I20260811 14:35:43.361270 9746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:45945 every 8 connection(s)
I20260811 14:35:43.361577 9523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:35:43.361876 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 9523
I20260811 14:35:43.361994 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:35:43.362206 2461 external_mini_cluster.cc:1428] Setting key 33460d10b00acc1c27ac9b690676caf5
Aug 11 14:35:43 dist-test-slave-twwt krb5kdc[9141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458943, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:43.372984 9273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:43.362330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:51759 (local address 127.2.103.126:46181)
0811 14:35:43.362395 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:43.362397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:43.362630 (+ 233us) server_negotiation.cc:407] Connection header received
0811 14:35:43.363349 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:43.363350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:43.363360 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:43.363412 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:43.364884 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.365793 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.366830 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.367070 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.367181 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:43.369877 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:43.369880 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:43.369880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:43.369889 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:43.371457 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:43.371953 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:43.371954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:43.371954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:43.372009 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:43.372313 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:43.372313 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:43.372314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:43.372425 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:43.372477 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:43.372840 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 14:35:43.372914 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
WARNING: no policy specified for kudu/127.2.103.67@KRBTEST.COM; defaulting to no policy
I20260811 14:35:43.373291 9749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:43.362309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46181 (local address 127.2.103.66:51759)
0811 14:35:43.362487 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:43.362537 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:43.363240 (+ 703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:43.363484 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:43.363488 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:43.363972 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:43.364747 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:43.364755 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.365904 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.365905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:43.366749 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:43.366752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.367195 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.367196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:43.367346 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:43.367924 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:43.367933 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:43.369779 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:43.371546 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:43.371555 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:43.371558 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:43.371879 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:43.372077 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:43.372078 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:43.372078 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:43.372246 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:43.372497 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:43.372505 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:43.372684 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 14:35:43.372823 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 14:35:43.373006 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
I20260811 14:35:43.373903 9747 heartbeater.cc:347] Connected to a master server at 127.2.103.126:46181
I20260811 14:35:43.374013 9747 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:43.374177 9747 heartbeater.cc:511] Master 127.2.103.126:46181 requested a full tablet report, sending...
I20260811 14:35:43.374583 9207 ts_manager.cc:194] Registered new tserver with Master: 695c1771631a441ba142e473f2d7dec1 (127.2.103.66:45945)
I20260811 14:35:43.374991 9207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:51759
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
I20260811 14:35:43.393412 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:46181
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:35351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:43.479899 9752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:43.480106 9752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:43.480131 9752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:43.481320 9752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:43.481369 9752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:43.481395 9752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:35:43.482964 9752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:35351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:46181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:43.483213 9752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:43.483456 9752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:43.488641 9764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:43.488636 9762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:43.488636 9766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:43.488826 9752 server_base.cc:1034] running on GCE node
I20260811 14:35:43.489133 9752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:43.489341 9752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:43.490484 9752 hybrid_clock.cc:648] HybridClock initialized: now 1786458943490470 us; error 28 us; skew 500 ppm
Aug 11 14:35:43 dist-test-slave-twwt krb5kdc[9141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458943, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:43.493155 9752 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:35:43.493533 9752 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:43.493585 9752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:43.493677 9752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:43.494776 9752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "7904b58d45174224a3d4dbb2661b3fa3"
format_stamp: "Formatted at 2026-08-11 14:35:43 on dist-test-slave-twwt"
server_key: "f3ef623b9f041565556a5ae28a28fb1a"
server_key_iv: "b23f48c1e3fe3f9b32236dea71bb66a9"
server_key_version: "encryptionkey@0"
I20260811 14:35:43.495096 9752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "7904b58d45174224a3d4dbb2661b3fa3"
format_stamp: "Formatted at 2026-08-11 14:35:43 on dist-test-slave-twwt"
server_key: "f3ef623b9f041565556a5ae28a28fb1a"
server_key_iv: "b23f48c1e3fe3f9b32236dea71bb66a9"
server_key_version: "encryptionkey@0"
I20260811 14:35:43.496380 9752 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:35:43.497066 9779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:43.497234 9752 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 14:35:43.497344 9752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "7904b58d45174224a3d4dbb2661b3fa3"
format_stamp: "Formatted at 2026-08-11 14:35:43 on dist-test-slave-twwt"
server_key: "f3ef623b9f041565556a5ae28a28fb1a"
server_key_iv: "b23f48c1e3fe3f9b32236dea71bb66a9"
server_key_version: "encryptionkey@0"
I20260811 14:35:43.497421 9752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:43.525274 9752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:43.527266 9752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:43.527436 9752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:43.527689 9752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:43.528036 9752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:43.528090 9752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:43.528133 9752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:43.528164 9752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:43.534204 9752 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:34991
I20260811 14:35:43.534247 9994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:34991 every 8 connection(s)
I20260811 14:35:43.534524 9752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:35:43.539757 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 9752
I20260811 14:35:43.539878 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:35:43.540073 2461 external_mini_cluster.cc:1428] Setting key d9c54811b52e3f4f7f4070c8a002d130
Aug 11 14:35:43 dist-test-slave-twwt krb5kdc[9141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458943, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:43.544274 9273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:43.535297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:50679 (local address 127.2.103.126:46181)
0811 14:35:43.535362 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:43.535364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:43.535552 (+ 188us) server_negotiation.cc:407] Connection header received
0811 14:35:43.536086 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:43.536088 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:43.536100 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:43.536164 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:43.537165 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.537696 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.538716 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.538909 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.538966 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:43.541557 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:43.541559 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:43.541559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:43.541568 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:43.542847 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:43.543241 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:43.543243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:43.543243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:43.543314 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:43.543576 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:43.543577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:43.543577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:43.543725 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:43.543790 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:43.544093 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 14:35:43.544182 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 14:35:43.544478 9998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:43.535268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46181 (local address 127.2.103.67:50679)
0811 14:35:43.535441 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:43.535478 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:43.535972 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:43.536246 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:43.536249 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:43.536523 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:43.537059 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:43.537064 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.537778 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.537778 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:43.538597 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:43.538599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.538983 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.538983 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:43.539090 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:43.539690 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:43.539699 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:43.541462 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:43.542925 (+ 1463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:43.542932 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:43.542935 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:43.543163 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:43.543387 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:43.543387 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:43.543388 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:43.543502 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:43.543780 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:43.543787 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:43.543952 (+ 165us) client_negotiation.cc:769] Sending connection context
0811 14:35:43.544060 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 14:35:43.544229 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260811 14:35:43.544991 9995 heartbeater.cc:347] Connected to a master server at 127.2.103.126:46181
I20260811 14:35:43.545110 9995 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:43.545276 9995 heartbeater.cc:511] Master 127.2.103.126:46181 requested a full tablet report, sending...
I20260811 14:35:43.545650 9207 ts_manager.cc:194] Registered new tserver with Master: 7904b58d45174224a3d4dbb2661b3fa3 (127.2.103.67:34991)
I20260811 14:35:43.545997 9207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:50679
I20260811 14:35:43.551824 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:35:43.560001 9273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:43.553370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35220 (local address 127.2.103.126:46181)
0811 14:35:43.553429 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:43.553431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:43.553542 (+ 111us) server_negotiation.cc:407] Connection header received
0811 14:35:43.553588 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:43.553589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:43.553599 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:43.553649 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:43.554190 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.554828 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.555631 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.555819 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.555879 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:43.557463 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:43.557464 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:43.557465 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:43.557472 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:43.558930 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:43.559250 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:43.559251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:43.559251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:43.559306 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:43.559472 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:43.559473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:43.559473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:43.559622 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:43.559700 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:43.559793 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 14:35:43.559878 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 14:35:43.562137 9207 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:35:43.563870 9207 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:35:43.570868 10017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:43.566593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53746 (local address 127.2.103.67:34991)
0811 14:35:43.566946 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:43.566951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:43.566962 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:35:43.567006 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:43.567009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:43.567028 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:43.567145 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:43.567354 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.568155 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.569611 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.570395 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.570481 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:43.570554 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:43.570738 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 14:35:43.570784 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":123,"threads_started":1}
I20260811 14:35:43.570855 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:43.566450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:34991 (local address 127.0.0.1:53746)
0811 14:35:43.566641 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:43.566685 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:43.566759 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:43.567152 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:43.567156 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:43.567158 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:43.567262 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:43.567265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.568255 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.568256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:43.569513 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:43.569516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.570494 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.570495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:43.570576 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:43.570590 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 14:35:43.570703 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 14:35:43.570752 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":66,"threads_started":1}
I20260811 14:35:43.571050 10022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:43.566919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40470 (local address 127.2.103.66:45945)
0811 14:35:43.567121 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:43.567126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:43.567137 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:35:43.567174 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:43.567176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:43.567191 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:43.567264 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:43.567463 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.568626 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.569960 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.570487 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.570557 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:43.570616 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:43.570923 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 14:35:43.570972 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":97,"threads_started":1}
I20260811 14:35:43.571074 10018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:43.566639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39198 (local address 127.2.103.65:44705)
0811 14:35:43.566848 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:43.566854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:43.566913 (+ 59us) server_negotiation.cc:407] Connection header received
0811 14:35:43.567002 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:43.567005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:43.567022 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:43.567109 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:43.567358 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.568567 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.569653 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.570443 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.570867 (+ 424us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:43.570936 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:43.570959 (+ 23us) server_negotiation.cc:299] Negotiation successful
0811 14:35:43.570996 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":92,"threads_started":1}
I20260811 14:35:43.571293 10016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:43.566713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:45945 (local address 127.0.0.1:40470)
0811 14:35:43.567014 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:43.567023 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:43.567061 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:43.567345 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:43.567346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:43.567348 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:43.567383 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:43.567385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.568709 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.568710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:43.569892 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:43.569894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.570681 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.570682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:43.570733 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:43.570736 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:35:43.570787 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 14:35:43.570824 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":63,"threads_started":1}
I20260811 14:35:43.570870 10021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:43.566494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:44705 (local address 127.0.0.1:39198)
0811 14:35:43.566850 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:43.566861 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:43.566926 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:43.567209 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:43.567211 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:43.567214 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:43.567275 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:43.567277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.568703 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.568705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:43.569528 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:43.569531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.570536 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.570538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:43.570631 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:43.570636 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:35:43.570685 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 14:35:43.570746 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":3,"thread_start_us":96,"threads_started":1}
I20260811 14:35:43.571772 9877 tablet_service.cc:1467] Processing CreateTablet for tablet 96080a0eba9a47e38bb543b81e7e23a8 (DEFAULT_TABLE table=test-table [id=914267aa7b0840c483acaedb4b5989be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:43.572180 9877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96080a0eba9a47e38bb543b81e7e23a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:43.572098 9425 tablet_service.cc:1467] Processing CreateTablet for tablet 96080a0eba9a47e38bb543b81e7e23a8 (DEFAULT_TABLE table=test-table [id=914267aa7b0840c483acaedb4b5989be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:43.572376 9425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96080a0eba9a47e38bb543b81e7e23a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:43.572268 9636 tablet_service.cc:1467] Processing CreateTablet for tablet 96080a0eba9a47e38bb543b81e7e23a8 (DEFAULT_TABLE table=test-table [id=914267aa7b0840c483acaedb4b5989be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:43.572580 9636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96080a0eba9a47e38bb543b81e7e23a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:43.575194 10024 tablet_bootstrap.cc:492] T 96080a0eba9a47e38bb543b81e7e23a8 P f66422f7706e4f6c8628571f196281cc: Bootstrap starting.
I20260811 14:35:43.575749 10026 tablet_bootstrap.cc:492] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3: Bootstrap starting.
I20260811 14:35:43.575881 10024 tablet_bootstrap.cc:654] T 96080a0eba9a47e38bb543b81e7e23a8 P f66422f7706e4f6c8628571f196281cc: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:43.576102 10028 tablet_bootstrap.cc:492] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1: Bootstrap starting.
I20260811 14:35:43.576642 10026 tablet_bootstrap.cc:654] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:43.576663 10028 tablet_bootstrap.cc:654] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:43.576660 10024 log.cc:824] T 96080a0eba9a47e38bb543b81e7e23a8 P f66422f7706e4f6c8628571f196281cc: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:43.577234 10026 log.cc:824] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:43.577347 10028 log.cc:824] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:43.577761 10024 tablet_bootstrap.cc:492] T 96080a0eba9a47e38bb543b81e7e23a8 P f66422f7706e4f6c8628571f196281cc: No bootstrap required, opened a new log
I20260811 14:35:43.577847 10024 ts_tablet_manager.cc:1397] T 96080a0eba9a47e38bb543b81e7e23a8 P f66422f7706e4f6c8628571f196281cc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 14:35:43.578080 10026 tablet_bootstrap.cc:492] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3: No bootstrap required, opened a new log
I20260811 14:35:43.578151 10026 ts_tablet_manager.cc:1397] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:35:43.578253 10028 tablet_bootstrap.cc:492] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1: No bootstrap required, opened a new log
I20260811 14:35:43.578301 10028 ts_tablet_manager.cc:1397] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:35:43.579463 10024 raft_consensus.cc:348] T 96080a0eba9a47e38bb543b81e7e23a8 P f66422f7706e4f6c8628571f196281cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7904b58d45174224a3d4dbb2661b3fa3" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 34991 } } peers { permanent_uuid: "f66422f7706e4f6c8628571f196281cc" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 44705 } } peers { permanent_uuid: "695c1771631a441ba142e473f2d7dec1" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45945 } }
I20260811 14:35:43.579627 10024 raft_consensus.cc:374] T 96080a0eba9a47e38bb543b81e7e23a8 P f66422f7706e4f6c8628571f196281cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:43.579690 10024 raft_consensus.cc:729] T 96080a0eba9a47e38bb543b81e7e23a8 P f66422f7706e4f6c8628571f196281cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f66422f7706e4f6c8628571f196281cc, State: Initialized, Role: FOLLOWER
I20260811 14:35:43.579866 10024 consensus_queue.cc:260] T 96080a0eba9a47e38bb543b81e7e23a8 P f66422f7706e4f6c8628571f196281cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7904b58d45174224a3d4dbb2661b3fa3" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 34991 } } peers { permanent_uuid: "f66422f7706e4f6c8628571f196281cc" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 44705 } } peers { permanent_uuid: "695c1771631a441ba142e473f2d7dec1" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45945 } }
I20260811 14:35:43.580190 9517 heartbeater.cc:503] Master 127.2.103.126:46181 was elected leader, sending a full tablet report...
I20260811 14:35:43.580195 10024 ts_tablet_manager.cc:1428] T 96080a0eba9a47e38bb543b81e7e23a8 P f66422f7706e4f6c8628571f196281cc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 14:35:43.580247 10026 raft_consensus.cc:348] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7904b58d45174224a3d4dbb2661b3fa3" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 34991 } } peers { permanent_uuid: "f66422f7706e4f6c8628571f196281cc" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 44705 } } peers { permanent_uuid: "695c1771631a441ba142e473f2d7dec1" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45945 } }
I20260811 14:35:43.580289 10028 raft_consensus.cc:348] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7904b58d45174224a3d4dbb2661b3fa3" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 34991 } } peers { permanent_uuid: "f66422f7706e4f6c8628571f196281cc" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 44705 } } peers { permanent_uuid: "695c1771631a441ba142e473f2d7dec1" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45945 } }
I20260811 14:35:43.580413 10026 raft_consensus.cc:374] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:43.580425 10028 raft_consensus.cc:374] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:43.580458 10028 raft_consensus.cc:729] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 695c1771631a441ba142e473f2d7dec1, State: Initialized, Role: FOLLOWER
I20260811 14:35:43.580456 10026 raft_consensus.cc:729] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7904b58d45174224a3d4dbb2661b3fa3, State: Initialized, Role: FOLLOWER
I20260811 14:35:43.580577 10026 consensus_queue.cc:260] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7904b58d45174224a3d4dbb2661b3fa3" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 34991 } } peers { permanent_uuid: "f66422f7706e4f6c8628571f196281cc" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 44705 } } peers { permanent_uuid: "695c1771631a441ba142e473f2d7dec1" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45945 } }
I20260811 14:35:43.580600 10028 consensus_queue.cc:260] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7904b58d45174224a3d4dbb2661b3fa3" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 34991 } } peers { permanent_uuid: "f66422f7706e4f6c8628571f196281cc" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 44705 } } peers { permanent_uuid: "695c1771631a441ba142e473f2d7dec1" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45945 } }
I20260811 14:35:43.580890 9995 heartbeater.cc:503] Master 127.2.103.126:46181 was elected leader, sending a full tablet report...
I20260811 14:35:43.581038 10028 ts_tablet_manager.cc:1428] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 14:35:43.581158 9747 heartbeater.cc:503] Master 127.2.103.126:46181 was elected leader, sending a full tablet report...
I20260811 14:35:43.580899 10026 ts_tablet_manager.cc:1428] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:35:43.607014 10040 raft_consensus.cc:482] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:35:43.607149 10040 raft_consensus.cc:504] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7904b58d45174224a3d4dbb2661b3fa3" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 34991 } } peers { permanent_uuid: "f66422f7706e4f6c8628571f196281cc" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 44705 } } peers { permanent_uuid: "695c1771631a441ba142e473f2d7dec1" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45945 } }
I20260811 14:35:43.607563 10040 leader_election.cc:288] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7904b58d45174224a3d4dbb2661b3fa3 (127.2.103.67:34991), f66422f7706e4f6c8628571f196281cc (127.2.103.65:44705)
I20260811 14:35:43.611552 10017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:43.607888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:33465 (local address 127.2.103.67:34991)
0811 14:35:43.607933 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:43.607936 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:43.608287 (+ 351us) server_negotiation.cc:407] Connection header received
0811 14:35:43.608321 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:43.608322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:43.608333 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:43.608392 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:43.608529 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.609210 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.610558 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.611159 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.611230 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:43.611240 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:43.611433 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 14:35:43.611476 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:35:43.611552 10018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:43.607808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:46851 (local address 127.2.103.65:44705)
0811 14:35:43.607864 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:43.607866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:43.607875 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:35:43.607910 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:43.607912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:43.607922 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:43.607985 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:43.608295 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.609075 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.610558 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.611149 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.611230 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:43.611240 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:43.611433 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 14:35:43.611476 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 14:35:43.611555 9749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:43.607735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:44705 (local address 127.2.103.66:46851)
0811 14:35:43.607785 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:43.607798 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:43.607848 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:43.607994 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:43.607995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:43.607997 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:43.608219 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:43.608222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.609167 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.609168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:43.610471 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:43.610473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.611242 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.611244 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:43.611358 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:43.611361 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:35:43.611414 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 14:35:43.611453 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":58}
I20260811 14:35:43.611562 10043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:43.607755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:34991 (local address 127.2.103.66:33465)
0811 14:35:43.607977 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:43.607987 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:43.608270 (+ 283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:43.608403 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:43.608404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:43.608406 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:43.608459 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:43.608460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.609278 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.609279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:43.610462 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:43.610466 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.611243 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.611244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:43.611331 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:43.611336 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:35:43.611403 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 14:35:43.611453 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":2,"thread_start_us":153,"threads_started":1}
I20260811 14:35:43.612021 9454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96080a0eba9a47e38bb543b81e7e23a8" candidate_uuid: "695c1771631a441ba142e473f2d7dec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f66422f7706e4f6c8628571f196281cc" is_pre_election: true
I20260811 14:35:43.612021 9904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96080a0eba9a47e38bb543b81e7e23a8" candidate_uuid: "695c1771631a441ba142e473f2d7dec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7904b58d45174224a3d4dbb2661b3fa3" is_pre_election: true
I20260811 14:35:43.612210 9454 raft_consensus.cc:2454] T 96080a0eba9a47e38bb543b81e7e23a8 P f66422f7706e4f6c8628571f196281cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 695c1771631a441ba142e473f2d7dec1 in term 0.
I20260811 14:35:43.612210 9904 raft_consensus.cc:2454] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 695c1771631a441ba142e473f2d7dec1 in term 0.
W20260811 14:35:43.612210 9748 tablet.cc:2369] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:35:43.612483 9552 leader_election.cc:302] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 695c1771631a441ba142e473f2d7dec1, f66422f7706e4f6c8628571f196281cc; no voters:
I20260811 14:35:43.612732 10040 raft_consensus.cc:2781] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:35:43.612818 10040 raft_consensus.cc:482] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:35:43.612887 10040 raft_consensus.cc:3037] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:43.613605 10040 raft_consensus.cc:504] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7904b58d45174224a3d4dbb2661b3fa3" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 34991 } } peers { permanent_uuid: "f66422f7706e4f6c8628571f196281cc" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 44705 } } peers { permanent_uuid: "695c1771631a441ba142e473f2d7dec1" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45945 } }
I20260811 14:35:43.613768 10040 leader_election.cc:288] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [CANDIDATE]: Term 1 election: Requested vote from peers 7904b58d45174224a3d4dbb2661b3fa3 (127.2.103.67:34991), f66422f7706e4f6c8628571f196281cc (127.2.103.65:44705)
I20260811 14:35:43.613957 9904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96080a0eba9a47e38bb543b81e7e23a8" candidate_uuid: "695c1771631a441ba142e473f2d7dec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7904b58d45174224a3d4dbb2661b3fa3"
I20260811 14:35:43.613969 9454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96080a0eba9a47e38bb543b81e7e23a8" candidate_uuid: "695c1771631a441ba142e473f2d7dec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f66422f7706e4f6c8628571f196281cc"
I20260811 14:35:43.614041 9904 raft_consensus.cc:3037] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:43.614060 9454 raft_consensus.cc:3037] T 96080a0eba9a47e38bb543b81e7e23a8 P f66422f7706e4f6c8628571f196281cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:43.614940 9904 raft_consensus.cc:2454] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 695c1771631a441ba142e473f2d7dec1 in term 1.
I20260811 14:35:43.614974 9454 raft_consensus.cc:2454] T 96080a0eba9a47e38bb543b81e7e23a8 P f66422f7706e4f6c8628571f196281cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 695c1771631a441ba142e473f2d7dec1 in term 1.
I20260811 14:35:43.615155 9552 leader_election.cc:302] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 695c1771631a441ba142e473f2d7dec1, f66422f7706e4f6c8628571f196281cc; no voters:
I20260811 14:35:43.615263 10040 raft_consensus.cc:2781] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:43.615434 10040 raft_consensus.cc:686] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [term 1 LEADER]: Becoming Leader. State: Replica: 695c1771631a441ba142e473f2d7dec1, State: Running, Role: LEADER
I20260811 14:35:43.615515 10040 consensus_queue.cc:237] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7904b58d45174224a3d4dbb2661b3fa3" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 34991 } } peers { permanent_uuid: "f66422f7706e4f6c8628571f196281cc" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 44705 } } peers { permanent_uuid: "695c1771631a441ba142e473f2d7dec1" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45945 } }
I20260811 14:35:43.616380 9207 catalog_manager.cc:5495] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 695c1771631a441ba142e473f2d7dec1 (127.2.103.66). New cstate: current_term: 1 leader_uuid: "695c1771631a441ba142e473f2d7dec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7904b58d45174224a3d4dbb2661b3fa3" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 34991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f66422f7706e4f6c8628571f196281cc" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 44705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "695c1771631a441ba142e473f2d7dec1" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45945 } health_report { overall_health: HEALTHY } } }
I20260811 14:35:43.631752 10022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:43.629372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40484 (local address 127.2.103.66:45945)
0811 14:35:43.629430 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:43.629432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:43.629458 (+ 26us) server_negotiation.cc:407] Connection header received
0811 14:35:43.629493 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:43.629494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:43.629512 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:43.629567 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:43.629702 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.630212 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.631005 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:43.631190 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:43.631245 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:43.631430 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:43.631532 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:43.631595 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:43.631626 (+ 31us) server_negotiation.cc:299] Negotiation successful
0811 14:35:43.631642 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 14:35:43.633687 9904 raft_consensus.cc:1261] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3 [term 1 FOLLOWER]: Refusing update from remote peer 695c1771631a441ba142e473f2d7dec1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:43.633697 9454 raft_consensus.cc:1261] T 96080a0eba9a47e38bb543b81e7e23a8 P f66422f7706e4f6c8628571f196281cc [term 1 FOLLOWER]: Refusing update from remote peer 695c1771631a441ba142e473f2d7dec1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:43.634086 10045 consensus_queue.cc:1035] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7904b58d45174224a3d4dbb2661b3fa3" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 34991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:43.634220 10040 consensus_queue.cc:1035] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f66422f7706e4f6c8628571f196281cc" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 44705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:43.636575 10053 mvcc.cc:205] Tried to move back new op lower bound from 7317335833120772096 to 7317335833049726976. Current Snapshot: MvccSnapshot[applied={T|T < 7317335833120772096}]
I20260811 14:35:43.637012 10058 mvcc.cc:205] Tried to move back new op lower bound from 7317335833120772096 to 7317335833049726976. Current Snapshot: MvccSnapshot[applied={T|T < 7317335833120772096}]
I20260811 14:35:43.637328 10055 mvcc.cc:205] Tried to move back new op lower bound from 7317335833120772096 to 7317335833049726976. Current Snapshot: MvccSnapshot[applied={T|T < 7317335833120772096 or (T in {7317335833120772096})}]
I20260811 14:35:43.639701 9205 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:43.639818 9205 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:43.640674 9205 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5430e27dcbe044beaee9c001b2289555: Sending DeleteTablet for 3 replicas of tablet 96080a0eba9a47e38bb543b81e7e23a8
I20260811 14:35:43.641038 9877 tablet_service.cc:1514] Processing DeleteTablet for tablet 96080a0eba9a47e38bb543b81e7e23a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:43 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:53746
I20260811 14:35:43.641047 9636 tablet_service.cc:1514] Processing DeleteTablet for tablet 96080a0eba9a47e38bb543b81e7e23a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:43 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:40470
I20260811 14:35:43.641085 9425 tablet_service.cc:1514] Processing DeleteTablet for tablet 96080a0eba9a47e38bb543b81e7e23a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:43 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:39198
I20260811 14:35:43.641412 10073 tablet_replica.cc:330] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3: stopping tablet replica
I20260811 14:35:43.641523 10073 raft_consensus.cc:2229] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:43.641637 10073 raft_consensus.cc:2258] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:43.641637 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 9317
I20260811 14:35:43.642128 10073 ts_tablet_manager.cc:1905] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:43.645092 10073 ts_tablet_manager.cc:1918] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:35:43.645177 10073 log.cc:1196] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/96080a0eba9a47e38bb543b81e7e23a8
I20260811 14:35:43.645490 10073 ts_tablet_manager.cc:1939] T 96080a0eba9a47e38bb543b81e7e23a8 P 7904b58d45174224a3d4dbb2661b3fa3: Deleting consensus metadata
I20260811 14:35:43.645679 10075 tablet_replica.cc:330] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1: stopping tablet replica
I20260811 14:35:43.645776 10075 raft_consensus.cc:2229] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:35:43.645905 10075 raft_consensus.cc:2258] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:43.647114 9189 catalog_manager.cc:4841] TS 7904b58d45174224a3d4dbb2661b3fa3 (127.2.103.67:34991): tablet 96080a0eba9a47e38bb543b81e7e23a8 (table test-table [id=914267aa7b0840c483acaedb4b5989be]) successfully deleted
I20260811 14:35:43.647768 10075 ts_tablet_manager.cc:1905] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:43.649407 10075 ts_tablet_manager.cc:1918] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:35:43.649484 10075 log.cc:1196] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/96080a0eba9a47e38bb543b81e7e23a8
I20260811 14:35:43.649765 10075 ts_tablet_manager.cc:1939] T 96080a0eba9a47e38bb543b81e7e23a8 P 695c1771631a441ba142e473f2d7dec1: Deleting consensus metadata
I20260811 14:35:43.650377 9187 catalog_manager.cc:4841] TS 695c1771631a441ba142e473f2d7dec1 (127.2.103.66:45945): tablet 96080a0eba9a47e38bb543b81e7e23a8 (table test-table [id=914267aa7b0840c483acaedb4b5989be]) successfully deleted
W20260811 14:35:43.657536 9187 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.103.65:44705 (error 108)
I20260811 14:35:43.657750 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 9523
I20260811 14:35:43.658672 10016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:43.658028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:44705 (local address 127.0.0.1:39204)
0811 14:35:43.658549 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:43.658596 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.65:44705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495}
W20260811 14:35:43.662545 9187 catalog_manager.cc:4584] TS f66422f7706e4f6c8628571f196281cc (127.2.103.65:44705): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 96080a0eba9a47e38bb543b81e7e23a8: Network error: Client connection negotiation failed: client connection to 127.2.103.65:44705: connect: Connection refused (error 111)
I20260811 14:35:43.670830 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 9752
I20260811 14:35:43.687644 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 9164
2026-08-11T14:35:43Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2260 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T14:35:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:43Z Disabled control of system clock
I20260811 14:35:43.723491 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:38859
--webserver_interface=127.2.103.126
--webserver_port=0
--builtin_ntp_servers=127.2.103.84:34727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:38859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 14:35:43.809574 10085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:43.809890 10085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:43.809917 10085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:43.811180 10085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:43.811232 10085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:43.811244 10085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:43.811257 10085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:43.811296 10085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:43.812762 10085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:34727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:38859
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:38859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:43.813050 10085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:43.813373 10085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:43.818571 10092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:43.818580 10094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:43.818667 10097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:43.818879 10085 server_base.cc:1034] running on GCE node
I20260811 14:35:43.819197 10085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:43.819481 10085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:43.820652 10085 hybrid_clock.cc:648] HybridClock initialized: now 1786458943820634 us; error 29 us; skew 500 ppm
I20260811 14:35:43.821961 10085 webserver.cc:460] Webserver started at http://127.2.103.126:41355/ using document root <none> and password file <none>
I20260811 14:35:43.822227 10085 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:43.822278 10085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:43.822464 10085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:43.823697 10085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "a031b53721544ed2ad6fd205224281a1"
format_stamp: "Formatted at 2026-08-11 14:35:43 on dist-test-slave-twwt"
server_key: "8b879946d8ff130f52df2a108b506873"
server_key_iv: "cf78a80bbf20974f6ff44c17f6bb21fb"
server_key_version: "encryptionkey@0"
I20260811 14:35:43.824122 10085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "a031b53721544ed2ad6fd205224281a1"
format_stamp: "Formatted at 2026-08-11 14:35:43 on dist-test-slave-twwt"
server_key: "8b879946d8ff130f52df2a108b506873"
server_key_iv: "cf78a80bbf20974f6ff44c17f6bb21fb"
server_key_version: "encryptionkey@0"
I20260811 14:35:43.825644 10085 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:43.826411 10111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:43.826644 10085 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:43.826750 10085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "a031b53721544ed2ad6fd205224281a1"
format_stamp: "Formatted at 2026-08-11 14:35:43 on dist-test-slave-twwt"
server_key: "8b879946d8ff130f52df2a108b506873"
server_key_iv: "cf78a80bbf20974f6ff44c17f6bb21fb"
server_key_version: "encryptionkey@0"
I20260811 14:35:43.826834 10085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:43.830305 10085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:43.830668 10085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:43.830818 10085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:43.835050 10085 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:38859
I20260811 14:35:43.835093 10195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:38859 every 8 connection(s)
I20260811 14:35:43.835501 10085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:43.836253 10196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:43.838240 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 10085
I20260811 14:35:43.838389 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:35:43.838580 2461 external_mini_cluster.cc:1428] Setting key a1adb36cf2d5392578f5003aa17a4259
I20260811 14:35:43.839807 10196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1: Bootstrap starting.
I20260811 14:35:43.840569 10196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:43.840955 10196 log.cc:824] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:43.841426 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:43.839213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52174 (local address 127.2.103.126:38859)
0811 14:35:43.839430 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:43.839438 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:43.839463 (+ 25us) server_negotiation.cc:407] Connection header received
0811 14:35:43.839937 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:43.839948 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:43.840009 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:43.840138 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:43.840894 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:43.840898 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:43.840898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:43.840902 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:43.840941 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:43.840951 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:43.841006 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:43.841206 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 14:35:43.841267 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":85,"threads_started":1}
I20260811 14:35:43.841825 10196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1: No bootstrap required, opened a new log
I20260811 14:35:43.843736 10196 raft_consensus.cc:348] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a031b53721544ed2ad6fd205224281a1" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38859 } }
I20260811 14:35:43.843887 10196 raft_consensus.cc:374] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:43.843910 10196 raft_consensus.cc:729] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a031b53721544ed2ad6fd205224281a1, State: Initialized, Role: FOLLOWER
I20260811 14:35:43.843997 10196 consensus_queue.cc:260] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a031b53721544ed2ad6fd205224281a1" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38859 } }
I20260811 14:35:43.844067 10196 raft_consensus.cc:388] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:43.844089 10196 raft_consensus.cc:482] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:43.844121 10196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:43.844630 10196 raft_consensus.cc:504] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a031b53721544ed2ad6fd205224281a1" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38859 } }
I20260811 14:35:43.844722 10196 leader_election.cc:302] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a031b53721544ed2ad6fd205224281a1; no voters:
I20260811 14:35:43.844853 10196 leader_election.cc:288] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:43.844908 10203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:43.845084 10203 raft_consensus.cc:686] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1 [term 1 LEADER]: Becoming Leader. State: Replica: a031b53721544ed2ad6fd205224281a1, State: Running, Role: LEADER
I20260811 14:35:43.845137 10196 sys_catalog.cc:564] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:43.845255 10203 consensus_queue.cc:237] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a031b53721544ed2ad6fd205224281a1" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38859 } }
I20260811 14:35:43.845649 10205 sys_catalog.cc:455] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a031b53721544ed2ad6fd205224281a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a031b53721544ed2ad6fd205224281a1" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38859 } } }
I20260811 14:35:43.845873 10205 sys_catalog.cc:458] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:43.845654 10206 sys_catalog.cc:455] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a031b53721544ed2ad6fd205224281a1. Latest consensus state: current_term: 1 leader_uuid: "a031b53721544ed2ad6fd205224281a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a031b53721544ed2ad6fd205224281a1" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38859 } } }
I20260811 14:35:43.846107 10206 sys_catalog.cc:458] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:43.846457 10214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:43.847044 10214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:43.848497 10214 catalog_manager.cc:1347] Generated new cluster ID: d117f7253e1040a6bc745a4aa83ea987
I20260811 14:35:43.848551 10214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:35:43.876415 10214 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:43.876641 10214 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:43.883994 10214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1: Generated new TSK 0
I20260811 14:35:43.884234 10214 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:35:43.896498 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38859
--builtin_ntp_servers=127.2.103.84:34727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 14:35:43.985566 10235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:43.985796 10235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:43.985821 10235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:43.987124 10235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:43.987174 10235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:43.987200 10235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:35:43.988657 10235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:34727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:43.988946 10235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:43.989276 10235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:43.994351 10250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:43.994406 10244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:43.994362 10246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:43.994644 10235 server_base.cc:1034] running on GCE node
I20260811 14:35:43.994906 10235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:43.995167 10235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:43.996315 10235 hybrid_clock.cc:648] HybridClock initialized: now 1786458943996301 us; error 31 us; skew 500 ppm
I20260811 14:35:43.997728 10235 webserver.cc:460] Webserver started at http://127.2.103.65:42007/ using document root <none> and password file <none>
I20260811 14:35:43.998028 10235 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:43.998081 10235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:43.998253 10235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:43.999416 10235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "4e59f99dceee4ee19f91f643b388f687"
format_stamp: "Formatted at 2026-08-11 14:35:43 on dist-test-slave-twwt"
server_key: "d2780ed05f26a8003feae7f47f557458"
server_key_iv: "a1fe7bb3ba33f83cbfeff468867e30c8"
server_key_version: "encryptionkey@0"
I20260811 14:35:43.999782 10235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "4e59f99dceee4ee19f91f643b388f687"
format_stamp: "Formatted at 2026-08-11 14:35:43 on dist-test-slave-twwt"
server_key: "d2780ed05f26a8003feae7f47f557458"
server_key_iv: "a1fe7bb3ba33f83cbfeff468867e30c8"
server_key_version: "encryptionkey@0"
I20260811 14:35:44.001191 10235 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 14:35:44.002022 10257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:44.002213 10235 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:44.002314 10235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "4e59f99dceee4ee19f91f643b388f687"
format_stamp: "Formatted at 2026-08-11 14:35:43 on dist-test-slave-twwt"
server_key: "d2780ed05f26a8003feae7f47f557458"
server_key_iv: "a1fe7bb3ba33f83cbfeff468867e30c8"
server_key_version: "encryptionkey@0"
I20260811 14:35:44.002393 10235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:44.005995 10235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:44.006279 10235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:44.006438 10235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:44.006695 10235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:44.007057 10235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:44.007091 10235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:44.007143 10235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:44.007169 10235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:44.013224 10235 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:42247
I20260811 14:35:44.013281 10434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:42247 every 8 connection(s)
I20260811 14:35:44.013674 10235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:35:44.016958 10437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.014456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:38859 (local address 127.2.103.65:49133)
0811 14:35:44.014697 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:44.014751 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:44.015308 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:44.015527 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:44.015531 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:44.015540 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:44.016444 (+ 904us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:35:44.016444 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:44.016475 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:35:44.016506 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:44.016664 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:44.016666 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:44.016669 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:35:44.016722 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 14:35:44.016773 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":108,"threads_started":1}
I20260811 14:35:44.016959 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.014482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:49133 (local address 127.2.103.126:38859)
0811 14:35:44.014528 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:44.014530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:44.014859 (+ 329us) server_negotiation.cc:407] Connection header received
0811 14:35:44.015406 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:44.015408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:44.015414 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:44.015469 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:44.016593 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:44.016594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:44.016594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:44.016599 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:44.016613 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:44.016617 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:44.016666 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:44.016874 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 14:35:44.016891 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 14:35:44.017488 10435 heartbeater.cc:347] Connected to a master server at 127.2.103.126:38859
I20260811 14:35:44.017589 10435 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:44.017752 10435 heartbeater.cc:511] Master 127.2.103.126:38859 requested a full tablet report, sending...
I20260811 14:35:44.018292 10136 ts_manager.cc:194] Registered new tserver with Master: 4e59f99dceee4ee19f91f643b388f687 (127.2.103.65:42247)
I20260811 14:35:44.022456 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 10235
I20260811 14:35:44.022562 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:35:44.022816 2461 external_mini_cluster.cc:1428] Setting key f85224fa750c822a15c0cdde557f5e72
I20260811 14:35:44.023998 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38859
--builtin_ntp_servers=127.2.103.84:34727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 14:35:44.111192 10440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:44.111457 10440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:44.111496 10440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:44.112742 10440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:44.112861 10440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:44.112917 10440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:35:44.114403 10440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:34727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:44.114722 10440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:44.115059 10440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:44.120637 10450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:44.120637 10452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:44.120633 10448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:44.120965 10440 server_base.cc:1034] running on GCE node
I20260811 14:35:44.121217 10440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:44.121492 10440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:44.122674 10440 hybrid_clock.cc:648] HybridClock initialized: now 1786458944122647 us; error 41 us; skew 500 ppm
I20260811 14:35:44.123986 10440 webserver.cc:460] Webserver started at http://127.2.103.66:41661/ using document root <none> and password file <none>
I20260811 14:35:44.124207 10440 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:44.124253 10440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:44.124367 10440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:44.125500 10440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "002f6791021f4d6e9514ae5c21cdade8"
format_stamp: "Formatted at 2026-08-11 14:35:44 on dist-test-slave-twwt"
server_key: "21444cd019902029972760794e6c1e62"
server_key_iv: "252bf023e69bf3c08ec4c2894db8e56f"
server_key_version: "encryptionkey@0"
I20260811 14:35:44.125833 10440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "002f6791021f4d6e9514ae5c21cdade8"
format_stamp: "Formatted at 2026-08-11 14:35:44 on dist-test-slave-twwt"
server_key: "21444cd019902029972760794e6c1e62"
server_key_iv: "252bf023e69bf3c08ec4c2894db8e56f"
server_key_version: "encryptionkey@0"
I20260811 14:35:44.127280 10440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:44.128077 10459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:44.128266 10440 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:44.128364 10440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "002f6791021f4d6e9514ae5c21cdade8"
format_stamp: "Formatted at 2026-08-11 14:35:44 on dist-test-slave-twwt"
server_key: "21444cd019902029972760794e6c1e62"
server_key_iv: "252bf023e69bf3c08ec4c2894db8e56f"
server_key_version: "encryptionkey@0"
I20260811 14:35:44.128427 10440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:44.132130 10440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:44.132388 10440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:44.132494 10440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:44.132664 10440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:44.132967 10440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:44.133000 10440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:44.133023 10440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:44.133035 10440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:44.138985 10440 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:45573
I20260811 14:35:44.139071 10645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:45573 every 8 connection(s)
I20260811 14:35:44.139442 10440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:35:44.140210 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 10440
I20260811 14:35:44.140322 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:35:44.140545 2461 external_mini_cluster.cc:1428] Setting key 0b6e66fa33ba0a03bd0d4a5364463448
I20260811 14:35:44.141877 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38859
--builtin_ntp_servers=127.2.103.84:34727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 14:35:44.143114 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.140280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:49551 (local address 127.2.103.126:38859)
0811 14:35:44.140338 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:44.140340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:44.140669 (+ 329us) server_negotiation.cc:407] Connection header received
0811 14:35:44.141336 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:44.141337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:44.141346 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:44.141419 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:44.142810 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:44.142813 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:44.142813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:44.142820 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:44.142841 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:44.142845 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:44.142907 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:44.142995 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 14:35:44.143014 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 14:35:44.143242 10654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.140289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:38859 (local address 127.2.103.66:49551)
0811 14:35:44.140513 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:44.140550 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:44.141208 (+ 658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:44.141491 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:44.141495 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:44.141504 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:44.142598 (+ 1094us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:35:44.142598 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:44.142688 (+ 90us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:35:44.142720 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:44.142920 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:44.142925 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:44.142928 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:35:44.142985 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 14:35:44.143041 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":105,"threads_started":1}
I20260811 14:35:44.143711 10647 heartbeater.cc:347] Connected to a master server at 127.2.103.126:38859
I20260811 14:35:44.143828 10647 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:44.143993 10647 heartbeater.cc:511] Master 127.2.103.126:38859 requested a full tablet report, sending...
I20260811 14:35:44.144411 10136 ts_manager.cc:194] Registered new tserver with Master: 002f6791021f4d6e9514ae5c21cdade8 (127.2.103.66:45573)
W20260811 14:35:44.233492 10657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:44.233716 10657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:44.233740 10657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:44.234942 10657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:44.234984 10657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:44.235013 10657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:35:44.236362 10657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:34727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:44.236570 10657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:44.237056 10657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:44.242442 10664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:44.242432 10667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:44.242432 10669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:44.242465 10657 server_base.cc:1034] running on GCE node
I20260811 14:35:44.242893 10657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:44.243175 10657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:44.244379 10657 hybrid_clock.cc:648] HybridClock initialized: now 1786458944244368 us; error 32 us; skew 500 ppm
I20260811 14:35:44.245853 10657 webserver.cc:460] Webserver started at http://127.2.103.67:45177/ using document root <none> and password file <none>
I20260811 14:35:44.246119 10657 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:44.246188 10657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:44.246376 10657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:44.247571 10657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "4d86fb114b4b4fb3b231ee10cad1fcb9"
format_stamp: "Formatted at 2026-08-11 14:35:44 on dist-test-slave-twwt"
server_key: "83889b5a15988160807469bd37d72e22"
server_key_iv: "b69ec5fe65000de0a65e156c062ce327"
server_key_version: "encryptionkey@0"
I20260811 14:35:44.247958 10657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "4d86fb114b4b4fb3b231ee10cad1fcb9"
format_stamp: "Formatted at 2026-08-11 14:35:44 on dist-test-slave-twwt"
server_key: "83889b5a15988160807469bd37d72e22"
server_key_iv: "b69ec5fe65000de0a65e156c062ce327"
server_key_version: "encryptionkey@0"
I20260811 14:35:44.249368 10657 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:35:44.250149 10684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:44.250375 10657 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:44.250471 10657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "4d86fb114b4b4fb3b231ee10cad1fcb9"
format_stamp: "Formatted at 2026-08-11 14:35:44 on dist-test-slave-twwt"
server_key: "83889b5a15988160807469bd37d72e22"
server_key_iv: "b69ec5fe65000de0a65e156c062ce327"
server_key_version: "encryptionkey@0"
I20260811 14:35:44.250567 10657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:44.253965 10657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:44.254249 10657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:44.254390 10657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:44.254652 10657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:44.255007 10657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:44.255059 10657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:44.255100 10657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:44.255126 10657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:44.261098 10657 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:35581
I20260811 14:35:44.261174 10889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:35581 every 8 connection(s)
I20260811 14:35:44.261514 10657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:35:44.265007 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.262403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:50241 (local address 127.2.103.126:38859)
0811 14:35:44.262474 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:44.262476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:44.262763 (+ 287us) server_negotiation.cc:407] Connection header received
0811 14:35:44.263262 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:44.263263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:44.263267 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:44.263330 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:44.264758 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:44.264760 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:44.264760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:44.264765 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:44.264780 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:44.264785 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:44.264851 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:44.264939 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 14:35:44.264951 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 14:35:44.265226 10894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.262373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:38859 (local address 127.2.103.67:50241)
0811 14:35:44.262584 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:44.262663 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:44.263175 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:44.263393 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:44.263397 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:44.263406 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:44.264580 (+ 1174us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:35:44.264581 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:44.264615 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:35:44.264650 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:44.264863 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:44.264868 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:44.264871 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:35:44.264938 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 14:35:44.264997 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":104,"threads_started":1}
I20260811 14:35:44.265833 10891 heartbeater.cc:347] Connected to a master server at 127.2.103.126:38859
I20260811 14:35:44.265936 10891 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:44.266109 10891 heartbeater.cc:511] Master 127.2.103.126:38859 requested a full tablet report, sending...
I20260811 14:35:44.266539 10136 ts_manager.cc:194] Registered new tserver with Master: 4d86fb114b4b4fb3b231ee10cad1fcb9 (127.2.103.67:35581)
I20260811 14:35:44.269757 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 10657
I20260811 14:35:44.269845 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:35:44.270043 2461 external_mini_cluster.cc:1428] Setting key a9a2b1703fb2ab4aaa5e43971dfd0408
I20260811 14:35:44.270905 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:35:44.273537 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.272410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52180 (local address 127.2.103.126:38859)
0811 14:35:44.272464 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:44.272466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:44.272602 (+ 136us) server_negotiation.cc:407] Connection header received
0811 14:35:44.272680 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:44.272680 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:44.272687 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:44.272747 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:44.273308 (+ 561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:44.273310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:44.273310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:44.273314 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:44.273328 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:44.273332 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:44.273382 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:44.273447 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 14:35:44.273462 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 14:35:44.275250 10136 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:35:44.276832 10136 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:35:44.281157 10915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.279189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:45573 (local address 127.0.0.1:45782)
0811 14:35:44.279394 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:44.279437 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:44.279501 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:44.280252 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:44.280253 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:44.280255 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:44.280683 (+ 428us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:35:44.280684 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:44.280702 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:35:44.280729 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:44.280976 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:44.280983 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:44.280986 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:35:44.281060 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 14:35:44.281075 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":112,"threads_started":1}
I20260811 14:35:44.281200 10918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.279464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45782 (local address 127.2.103.66:45573)
0811 14:35:44.279661 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:44.279666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:44.279678 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:35:44.279708 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:44.279711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:44.279720 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:44.279812 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:44.280806 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:44.280818 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:44.280818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:44.280824 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:44.280863 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:44.280899 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:44.280968 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:44.281067 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 14:35:44.281085 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":95,"threads_started":1}
I20260811 14:35:44.281314 10917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.279469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34180 (local address 127.2.103.65:42247)
0811 14:35:44.279661 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:44.279666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:44.279678 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:35:44.279709 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:44.279711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:44.279720 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:44.279812 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:44.280859 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:44.280871 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:44.280872 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:44.280877 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:44.280913 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:44.280945 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:44.281015 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:44.281194 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 14:35:44.281211 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":90,"threads_started":1}
I20260811 14:35:44.281437 10916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.279286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39472 (local address 127.2.103.67:35581)
0811 14:35:44.280321 (+ 1035us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:44.280325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:44.280332 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:35:44.280354 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:44.280355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:44.280361 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:44.280451 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:44.280997 (+ 546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:44.281004 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:44.281004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:44.281008 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:44.281035 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:44.281061 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:44.281113 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:44.281331 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 14:35:44.281348 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1003,"thread_start_us":94,"threads_started":1}
I20260811 14:35:44.281270 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.279199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:35581 (local address 127.0.0.1:39472)
0811 14:35:44.280179 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:44.280188 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:44.280225 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:44.280453 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:44.280454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:44.280456 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:44.280902 (+ 446us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:35:44.280902 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:44.280908 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:35:44.280917 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:44.281137 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:44.281138 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:44.281139 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 14:35:44.281180 (+ 41us) client_negotiation.cc:240] Negotiation successful
0811 14:35:44.281195 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":962,"thread_start_us":117,"threads_started":1}
I20260811 14:35:44.281159 10910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.279197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:42247 (local address 127.0.0.1:34180)
0811 14:35:44.279464 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:44.279473 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:44.279511 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:44.279857 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:44.279865 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:44.279873 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:44.280759 (+ 886us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:35:44.280759 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:44.280767 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:35:44.280782 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:44.281052 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:44.281053 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:44.281054 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 14:35:44.281098 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 14:35:44.281108 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":73,"threads_started":1}
I20260811 14:35:44.282356 10769 tablet_service.cc:1467] Processing CreateTablet for tablet 93eab7e3ef324da1a378b6cbe4d8e12d (DEFAULT_TABLE table=test-table [id=1eea506b202d40959c0c2ec72aeb908a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:44.282358 10555 tablet_service.cc:1467] Processing CreateTablet for tablet 93eab7e3ef324da1a378b6cbe4d8e12d (DEFAULT_TABLE table=test-table [id=1eea506b202d40959c0c2ec72aeb908a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:44.282757 10555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93eab7e3ef324da1a378b6cbe4d8e12d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:44.282760 10769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93eab7e3ef324da1a378b6cbe4d8e12d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:44.285498 10919 tablet_bootstrap.cc:492] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9: Bootstrap starting.
I20260811 14:35:44.286139 10919 tablet_bootstrap.cc:654] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:44.286077 10326 tablet_service.cc:1467] Processing CreateTablet for tablet 93eab7e3ef324da1a378b6cbe4d8e12d (DEFAULT_TABLE table=test-table [id=1eea506b202d40959c0c2ec72aeb908a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:44.286418 10326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93eab7e3ef324da1a378b6cbe4d8e12d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:44.286456 10919 log.cc:824] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:44.286777 10920 tablet_bootstrap.cc:492] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8: Bootstrap starting.
I20260811 14:35:44.287452 10920 tablet_bootstrap.cc:654] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:44.287518 10919 tablet_bootstrap.cc:492] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9: No bootstrap required, opened a new log
I20260811 14:35:44.287590 10919 ts_tablet_manager.cc:1397] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 14:35:44.287783 10920 log.cc:824] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:44.288610 10920 tablet_bootstrap.cc:492] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8: No bootstrap required, opened a new log
I20260811 14:35:44.288684 10920 ts_tablet_manager.cc:1397] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:35:44.289556 10926 tablet_bootstrap.cc:492] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4e59f99dceee4ee19f91f643b388f687: Bootstrap starting.
I20260811 14:35:44.290171 10926 tablet_bootstrap.cc:654] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4e59f99dceee4ee19f91f643b388f687: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:44.290480 10926 log.cc:824] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4e59f99dceee4ee19f91f643b388f687: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:44.290311 10919 raft_consensus.cc:348] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "002f6791021f4d6e9514ae5c21cdade8" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45573 } } peers { permanent_uuid: "4d86fb114b4b4fb3b231ee10cad1fcb9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35581 } } peers { permanent_uuid: "4e59f99dceee4ee19f91f643b388f687" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 42247 } }
I20260811 14:35:44.290910 10919 raft_consensus.cc:374] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:44.290978 10919 raft_consensus.cc:729] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d86fb114b4b4fb3b231ee10cad1fcb9, State: Initialized, Role: FOLLOWER
I20260811 14:35:44.291153 10919 consensus_queue.cc:260] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "002f6791021f4d6e9514ae5c21cdade8" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45573 } } peers { permanent_uuid: "4d86fb114b4b4fb3b231ee10cad1fcb9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35581 } } peers { permanent_uuid: "4e59f99dceee4ee19f91f643b388f687" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 42247 } }
I20260811 14:35:44.291301 10926 tablet_bootstrap.cc:492] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4e59f99dceee4ee19f91f643b388f687: No bootstrap required, opened a new log
I20260811 14:35:44.291366 10926 ts_tablet_manager.cc:1397] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4e59f99dceee4ee19f91f643b388f687: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 14:35:44.291438 10919 ts_tablet_manager.cc:1428] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260811 14:35:44.291579 10891 heartbeater.cc:503] Master 127.2.103.126:38859 was elected leader, sending a full tablet report...
I20260811 14:35:44.291446 10920 raft_consensus.cc:348] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "002f6791021f4d6e9514ae5c21cdade8" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45573 } } peers { permanent_uuid: "4d86fb114b4b4fb3b231ee10cad1fcb9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35581 } } peers { permanent_uuid: "4e59f99dceee4ee19f91f643b388f687" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 42247 } }
I20260811 14:35:44.291901 10920 raft_consensus.cc:374] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:44.291955 10920 raft_consensus.cc:729] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 002f6791021f4d6e9514ae5c21cdade8, State: Initialized, Role: FOLLOWER
I20260811 14:35:44.292141 10920 consensus_queue.cc:260] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "002f6791021f4d6e9514ae5c21cdade8" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45573 } } peers { permanent_uuid: "4d86fb114b4b4fb3b231ee10cad1fcb9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35581 } } peers { permanent_uuid: "4e59f99dceee4ee19f91f643b388f687" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 42247 } }
I20260811 14:35:44.292449 10920 ts_tablet_manager.cc:1428] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 14:35:44.292495 10647 heartbeater.cc:503] Master 127.2.103.126:38859 was elected leader, sending a full tablet report...
I20260811 14:35:44.294045 10926 raft_consensus.cc:348] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4e59f99dceee4ee19f91f643b388f687 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "002f6791021f4d6e9514ae5c21cdade8" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45573 } } peers { permanent_uuid: "4d86fb114b4b4fb3b231ee10cad1fcb9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35581 } } peers { permanent_uuid: "4e59f99dceee4ee19f91f643b388f687" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 42247 } }
I20260811 14:35:44.294214 10926 raft_consensus.cc:374] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4e59f99dceee4ee19f91f643b388f687 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:44.294262 10926 raft_consensus.cc:729] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4e59f99dceee4ee19f91f643b388f687 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e59f99dceee4ee19f91f643b388f687, State: Initialized, Role: FOLLOWER
I20260811 14:35:44.294384 10926 consensus_queue.cc:260] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4e59f99dceee4ee19f91f643b388f687 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "002f6791021f4d6e9514ae5c21cdade8" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45573 } } peers { permanent_uuid: "4d86fb114b4b4fb3b231ee10cad1fcb9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35581 } } peers { permanent_uuid: "4e59f99dceee4ee19f91f643b388f687" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 42247 } }
I20260811 14:35:44.294759 10926 ts_tablet_manager.cc:1428] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4e59f99dceee4ee19f91f643b388f687: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260811 14:35:44.294768 10435 heartbeater.cc:503] Master 127.2.103.126:38859 was elected leader, sending a full tablet report...
W20260811 14:35:44.390070 10650 tablet.cc:2369] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:35:44.416709 10933 raft_consensus.cc:482] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:35:44.416843 10933 raft_consensus.cc:504] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "002f6791021f4d6e9514ae5c21cdade8" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45573 } } peers { permanent_uuid: "4d86fb114b4b4fb3b231ee10cad1fcb9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35581 } } peers { permanent_uuid: "4e59f99dceee4ee19f91f643b388f687" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 42247 } }
I20260811 14:35:44.417275 10933 leader_election.cc:288] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d86fb114b4b4fb3b231ee10cad1fcb9 (127.2.103.67:35581), 4e59f99dceee4ee19f91f643b388f687 (127.2.103.65:42247)
I20260811 14:35:44.418831 10917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.417650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:42509 (local address 127.2.103.65:42247)
0811 14:35:44.417708 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:44.417763 (+ 55us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:44.417775 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:35:44.417811 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:44.417812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:44.417819 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:44.417880 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:44.418545 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:44.418546 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:44.418546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:44.418551 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:44.418568 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:44.418573 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:44.418692 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:44.418749 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 14:35:44.418762 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 14:35:44.418831 10654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.417525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:42247 (local address 127.2.103.66:42509)
0811 14:35:44.417579 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:44.417593 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:44.417647 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:44.417916 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:44.417917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:44.417919 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:44.418441 (+ 522us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:35:44.418442 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:44.418450 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:35:44.418461 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:44.418689 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:44.418691 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:44.418691 (+ 0us) client_negotiation.cc:769] Sending connection context
0811 14:35:44.418743 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 14:35:44.418760 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 14:35:44.419062 10916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.417640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:49979 (local address 127.2.103.67:35581)
0811 14:35:44.417693 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:44.417695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:44.417836 (+ 141us) server_negotiation.cc:407] Connection header received
0811 14:35:44.417926 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:44.417927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:44.417933 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:44.418004 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:44.418811 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:44.418812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:44.418813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:44.418818 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:44.418834 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:44.418838 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:44.418911 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:44.418980 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 14:35:44.418996 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 14:35:44.419112 10938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.417533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:35581 (local address 127.2.103.66:49979)
0811 14:35:44.417749 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:44.417760 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:44.417833 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:44.418018 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:44.418019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:44.418035 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:44.418660 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:35:44.418661 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:44.418670 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:35:44.418686 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:44.418908 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:44.418909 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:44.418910 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 14:35:44.418973 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 14:35:44.418989 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":25,"thread_start_us":109,"threads_started":1}
I20260811 14:35:44.419224 10360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93eab7e3ef324da1a378b6cbe4d8e12d" candidate_uuid: "002f6791021f4d6e9514ae5c21cdade8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e59f99dceee4ee19f91f643b388f687" is_pre_election: true
I20260811 14:35:44.419435 10360 raft_consensus.cc:2454] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4e59f99dceee4ee19f91f643b388f687 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 002f6791021f4d6e9514ae5c21cdade8 in term 0.
I20260811 14:35:44.419466 10815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93eab7e3ef324da1a378b6cbe4d8e12d" candidate_uuid: "002f6791021f4d6e9514ae5c21cdade8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d86fb114b4b4fb3b231ee10cad1fcb9" is_pre_election: true
I20260811 14:35:44.419605 10815 raft_consensus.cc:2454] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 002f6791021f4d6e9514ae5c21cdade8 in term 0.
I20260811 14:35:44.419708 10464 leader_election.cc:302] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 002f6791021f4d6e9514ae5c21cdade8, 4e59f99dceee4ee19f91f643b388f687; no voters:
I20260811 14:35:44.419854 10933 raft_consensus.cc:2781] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:35:44.419910 10933 raft_consensus.cc:482] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:35:44.419934 10933 raft_consensus.cc:3037] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:44.420683 10933 raft_consensus.cc:504] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "002f6791021f4d6e9514ae5c21cdade8" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45573 } } peers { permanent_uuid: "4d86fb114b4b4fb3b231ee10cad1fcb9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35581 } } peers { permanent_uuid: "4e59f99dceee4ee19f91f643b388f687" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 42247 } }
I20260811 14:35:44.420903 10933 leader_election.cc:288] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [CANDIDATE]: Term 1 election: Requested vote from peers 4d86fb114b4b4fb3b231ee10cad1fcb9 (127.2.103.67:35581), 4e59f99dceee4ee19f91f643b388f687 (127.2.103.65:42247)
I20260811 14:35:44.421030 10360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93eab7e3ef324da1a378b6cbe4d8e12d" candidate_uuid: "002f6791021f4d6e9514ae5c21cdade8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e59f99dceee4ee19f91f643b388f687"
I20260811 14:35:44.421074 10815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93eab7e3ef324da1a378b6cbe4d8e12d" candidate_uuid: "002f6791021f4d6e9514ae5c21cdade8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d86fb114b4b4fb3b231ee10cad1fcb9"
I20260811 14:35:44.421123 10360 raft_consensus.cc:3037] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4e59f99dceee4ee19f91f643b388f687 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:44.421181 10815 raft_consensus.cc:3037] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:44.421913 10360 raft_consensus.cc:2454] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4e59f99dceee4ee19f91f643b388f687 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 002f6791021f4d6e9514ae5c21cdade8 in term 1.
I20260811 14:35:44.421937 10815 raft_consensus.cc:2454] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 002f6791021f4d6e9514ae5c21cdade8 in term 1.
I20260811 14:35:44.422091 10464 leader_election.cc:302] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 002f6791021f4d6e9514ae5c21cdade8, 4e59f99dceee4ee19f91f643b388f687; no voters:
I20260811 14:35:44.422179 10933 raft_consensus.cc:2781] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:44.422358 10933 raft_consensus.cc:686] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [term 1 LEADER]: Becoming Leader. State: Replica: 002f6791021f4d6e9514ae5c21cdade8, State: Running, Role: LEADER
I20260811 14:35:44.422470 10933 consensus_queue.cc:237] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "002f6791021f4d6e9514ae5c21cdade8" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45573 } } peers { permanent_uuid: "4d86fb114b4b4fb3b231ee10cad1fcb9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35581 } } peers { permanent_uuid: "4e59f99dceee4ee19f91f643b388f687" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 42247 } }
I20260811 14:35:44.423353 10136 catalog_manager.cc:5495] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 002f6791021f4d6e9514ae5c21cdade8 (127.2.103.66). New cstate: current_term: 1 leader_uuid: "002f6791021f4d6e9514ae5c21cdade8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "002f6791021f4d6e9514ae5c21cdade8" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d86fb114b4b4fb3b231ee10cad1fcb9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e59f99dceee4ee19f91f643b388f687" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 42247 } health_report { overall_health: UNKNOWN } } }
I20260811 14:35:44.463106 10918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.462069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45792 (local address 127.2.103.66:45573)
0811 14:35:44.462138 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:44.462140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:44.462148 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:35:44.462188 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:44.462189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:44.462202 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:44.462253 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:44.462918 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:44.462919 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:44.462920 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:44.462924 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:44.462940 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:44.462944 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:44.462997 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:44.463035 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 14:35:44.463048 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 14:35:44.465097 10360 raft_consensus.cc:1261] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4e59f99dceee4ee19f91f643b388f687 [term 1 FOLLOWER]: Refusing update from remote peer 002f6791021f4d6e9514ae5c21cdade8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:44.465098 10815 raft_consensus.cc:1261] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9 [term 1 FOLLOWER]: Refusing update from remote peer 002f6791021f4d6e9514ae5c21cdade8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:44.465524 10933 consensus_queue.cc:1035] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d86fb114b4b4fb3b231ee10cad1fcb9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:44.465730 10939 consensus_queue.cc:1035] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e59f99dceee4ee19f91f643b388f687" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:44.469681 10948 mvcc.cc:205] Tried to move back new op lower bound from 7317335836526002176 to 7317335836355137536. Current Snapshot: MvccSnapshot[applied={T|T < 7317335836526002176}]
I20260811 14:35:44.470839 10134 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:44.470952 10134 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:44.471786 10134 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a031b53721544ed2ad6fd205224281a1: Sending DeleteTablet for 3 replicas of tablet 93eab7e3ef324da1a378b6cbe4d8e12d
I20260811 14:35:44.472146 10555 tablet_service.cc:1514] Processing DeleteTablet for tablet 93eab7e3ef324da1a378b6cbe4d8e12d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:44 UTC) from {username='slave'} at 127.0.0.1:45782
I20260811 14:35:44.472218 10326 tablet_service.cc:1514] Processing DeleteTablet for tablet 93eab7e3ef324da1a378b6cbe4d8e12d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:44 UTC) from {username='slave'} at 127.0.0.1:34180
I20260811 14:35:44.472258 10769 tablet_service.cc:1514] Processing DeleteTablet for tablet 93eab7e3ef324da1a378b6cbe4d8e12d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:44 UTC) from {username='slave'} at 127.0.0.1:39472
I20260811 14:35:44.472782 10963 tablet_replica.cc:330] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8: stopping tablet replica
I20260811 14:35:44.472797 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 10235
I20260811 14:35:44.472940 10963 raft_consensus.cc:2229] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:35:44.480158 10963 raft_consensus.cc:2258] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:44.480487 10968 tablet_replica.cc:330] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9: stopping tablet replica
I20260811 14:35:44.480683 10968 raft_consensus.cc:2229] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:44.480886 10968 raft_consensus.cc:2258] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:44.482168 10968 ts_tablet_manager.cc:1905] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:44.482452 10963 ts_tablet_manager.cc:1905] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:44.484227 10968 ts_tablet_manager.cc:1918] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:35:44.484405 10968 log.cc:1196] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/93eab7e3ef324da1a378b6cbe4d8e12d
I20260811 14:35:44.484691 10963 ts_tablet_manager.cc:1918] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:35:44.484802 10963 log.cc:1196] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/93eab7e3ef324da1a378b6cbe4d8e12d
I20260811 14:35:44.485199 10968 ts_tablet_manager.cc:1939] T 93eab7e3ef324da1a378b6cbe4d8e12d P 4d86fb114b4b4fb3b231ee10cad1fcb9: Deleting consensus metadata
I20260811 14:35:44.485203 10963 ts_tablet_manager.cc:1939] T 93eab7e3ef324da1a378b6cbe4d8e12d P 002f6791021f4d6e9514ae5c21cdade8: Deleting consensus metadata
I20260811 14:35:44.485721 10113 catalog_manager.cc:4841] TS 002f6791021f4d6e9514ae5c21cdade8 (127.2.103.66:45573): tablet 93eab7e3ef324da1a378b6cbe4d8e12d (table test-table [id=1eea506b202d40959c0c2ec72aeb908a]) successfully deleted
I20260811 14:35:44.486135 10116 catalog_manager.cc:4841] TS 4d86fb114b4b4fb3b231ee10cad1fcb9 (127.2.103.67:35581): tablet 93eab7e3ef324da1a378b6cbe4d8e12d (table test-table [id=1eea506b202d40959c0c2ec72aeb908a]) successfully deleted
W20260811 14:35:44.490134 10115 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.103.65:42247 (error 108)
I20260811 14:35:44.490468 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 10440
I20260811 14:35:44.490726 10910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.490567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:42247 (local address 127.0.0.1:34192)
0811 14:35:44.490639 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:44.490684 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.65:42247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 14:35:44.498332 10115 catalog_manager.cc:4584] TS 4e59f99dceee4ee19f91f643b388f687 (127.2.103.65:42247): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 93eab7e3ef324da1a378b6cbe4d8e12d: Network error: Client connection negotiation failed: client connection to 127.2.103.65:42247: connect: Connection refused (error 111)
I20260811 14:35:44.503197 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 10657
I20260811 14:35:44.521549 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 10085
2026-08-11T14:35:44Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (838 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 14:35:44.554157 2461 mini_oidc.cc:152] Starting JWKS server
I20260811 14:35:44.555394 2461 webserver.cc:460] Webserver started at https://127.0.0.1:36577/ using document root <none> and password file <none>
I20260811 14:35:44.555502 2461 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 14:35:44.555681 2461 webserver.cc:460] Webserver started at http://127.0.0.1:36845/ using document root <none> and password file <none>
2026-08-11T14:35:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:44Z Disabled control of system clock
I20260811 14:35:44.565816 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:41565
--webserver_interface=127.2.103.126
--webserver_port=0
--builtin_ntp_servers=127.2.103.84:39625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:41565
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36577/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458919184199-2461-0/testchainca.cert with env {}
W20260811 14:35:44.651543 10982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:44.651767 10982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:44.651791 10982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:44.653047 10982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:44.653095 10982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:44.653108 10982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:44.653121 10982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:44.653134 10982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 14:35:44.653147 10982 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 14:35:44.653158 10982 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36577/jwks.json
I20260811 14:35:44.654929 10982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:39625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:41565
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:41565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36577/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458919184199-2461-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:44.655224 10982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:44.655475 10982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:44.660786 10989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:44.660769 10991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:44.660786 10995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:44.661062 10982 server_base.cc:1034] running on GCE node
I20260811 14:35:44.661312 10982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:44.661645 10982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:44.662824 10982 hybrid_clock.cc:648] HybridClock initialized: now 1786458944662802 us; error 38 us; skew 500 ppm
I20260811 14:35:44.664142 10982 webserver.cc:460] Webserver started at http://127.2.103.126:45545/ using document root <none> and password file <none>
I20260811 14:35:44.664422 10982 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:44.664494 10982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:44.664633 10982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:44.665728 10982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "b88b65ee582747f1b9b811d11c65624b"
format_stamp: "Formatted at 2026-08-11 14:35:44 on dist-test-slave-twwt"
server_key: "22d6975fc2a948e9336a29b17ec7f5a8"
server_key_iv: "4c7d93357a2d9c8f7137b6deaa98f746"
server_key_version: "encryptionkey@0"
I20260811 14:35:44.666103 10982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "b88b65ee582747f1b9b811d11c65624b"
format_stamp: "Formatted at 2026-08-11 14:35:44 on dist-test-slave-twwt"
server_key: "22d6975fc2a948e9336a29b17ec7f5a8"
server_key_iv: "4c7d93357a2d9c8f7137b6deaa98f746"
server_key_version: "encryptionkey@0"
I20260811 14:35:44.667484 10982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:35:44.668293 11005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:44.668474 10982 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:35:44.668584 10982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "b88b65ee582747f1b9b811d11c65624b"
format_stamp: "Formatted at 2026-08-11 14:35:44 on dist-test-slave-twwt"
server_key: "22d6975fc2a948e9336a29b17ec7f5a8"
server_key_iv: "4c7d93357a2d9c8f7137b6deaa98f746"
server_key_version: "encryptionkey@0"
I20260811 14:35:44.668658 10982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:44.678843 10982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:44.679184 10982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:44.679337 10982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:44.683801 10982 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:41565
I20260811 14:35:44.683841 11104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:41565 every 8 connection(s)
I20260811 14:35:44.684239 10982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:44.685256 11107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:44.687758 11107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b: Bootstrap starting.
I20260811 14:35:44.688402 11107 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:44.688715 11107 log.cc:824] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:44.689466 11107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b: No bootstrap required, opened a new log
I20260811 14:35:44.690964 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 10982
I20260811 14:35:44.691080 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:35:44.691118 11107 raft_consensus.cc:348] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88b65ee582747f1b9b811d11c65624b" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41565 } }
I20260811 14:35:44.691267 2461 external_mini_cluster.cc:1428] Setting key 08fcbd75e88362c31940039b54eddf82
I20260811 14:35:44.691291 11107 raft_consensus.cc:374] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:44.691330 11107 raft_consensus.cc:729] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b88b65ee582747f1b9b811d11c65624b, State: Initialized, Role: FOLLOWER
I20260811 14:35:44.691521 11107 consensus_queue.cc:260] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88b65ee582747f1b9b811d11c65624b" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41565 } }
I20260811 14:35:44.691623 11107 raft_consensus.cc:388] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:44.691690 11107 raft_consensus.cc:482] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:44.691740 11107 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:44.692307 11107 raft_consensus.cc:504] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88b65ee582747f1b9b811d11c65624b" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41565 } }
I20260811 14:35:44.692438 11107 leader_election.cc:302] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b88b65ee582747f1b9b811d11c65624b; no voters:
I20260811 14:35:44.692610 11107 leader_election.cc:288] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:44.692642 11115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:44.692821 11115 raft_consensus.cc:686] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b [term 1 LEADER]: Becoming Leader. State: Replica: b88b65ee582747f1b9b811d11c65624b, State: Running, Role: LEADER
I20260811 14:35:44.692916 11107 sys_catalog.cc:564] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:44.692919 11115 consensus_queue.cc:237] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88b65ee582747f1b9b811d11c65624b" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41565 } }
I20260811 14:35:44.693364 11118 sys_catalog.cc:455] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b88b65ee582747f1b9b811d11c65624b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88b65ee582747f1b9b811d11c65624b" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41565 } } }
I20260811 14:35:44.693454 11118 sys_catalog.cc:458] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:44.693616 11120 sys_catalog.cc:455] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b88b65ee582747f1b9b811d11c65624b. Latest consensus state: current_term: 1 leader_uuid: "b88b65ee582747f1b9b811d11c65624b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88b65ee582747f1b9b811d11c65624b" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41565 } } }
I20260811 14:35:44.693693 11120 sys_catalog.cc:458] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:44.693825 11129 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:44.694353 11129 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:44.696075 11129 catalog_manager.cc:1347] Generated new cluster ID: 98a4f77fe7f74faa9feda1f84ab6155d
I20260811 14:35:44.696132 11129 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:35:44.697893 11114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.691746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47776 (local address 127.2.103.126:41565)
0811 14:35:44.692006 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:44.692020 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:44.692046 (+ 26us) server_negotiation.cc:407] Connection header received
0811 14:35:44.692151 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:44.692157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:44.692217 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:44.692354 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:44.692614 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:44.694499 (+ 1885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:44.695788 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:44.696351 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:44.696494 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:44.697219 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:44.697223 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:44.697223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:44.697433 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:44.697465 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:44.697475 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:44.697547 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:44.697622 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 14:35:44.697701 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":92,"threads_started":1}
I20260811 14:35:44.699520 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:35:44.708889 11129 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:44.709587 11129 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:44.714821 11129 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b88b65ee582747f1b9b811d11c65624b: Generated new TSK 0
I20260811 14:35:44.715018 11129 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 14:35:44.746709 11160 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 14:35:44.746869 11160 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260811 14:35:44.747390 11159 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:44.723347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:41565 (local address 127.0.0.1:47786)
0811 14:35:44.723567 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:44.723579 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:44.723655 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:44.723863 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:44.723864 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:44.723867 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0811 14:35:44.723947 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:44.723949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:44.724590 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:44.724591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:44.725585 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:44.725588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:44.725941 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:44.725942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:44.726008 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:44.726032 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 14:35:44.746950 (+ 20918us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36577/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 14:35:44.747058 (+ 108us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.103.126:41565: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36577/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":114,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.2.103.126:41565: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36577/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 14:35:44.747746 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 10982
I20260811 14:35:44.747936 11114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.723417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47786 (local address 127.2.103.126:41565)
0811 14:35:44.723488 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:44.723490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:44.723645 (+ 155us) server_negotiation.cc:407] Connection header received
0811 14:35:44.723753 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:44.723754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:44.723766 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:44.723836 (+ 70us) server_negotiation.cc:226] Negotiated authn=JWT
0811 14:35:44.724019 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:44.724515 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:44.725674 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:44.725866 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:44.725921 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:44.726188 (+ 267us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 14:35:44.746708 (+ 20520us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36577/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 14:35:44.747823 (+ 1115us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47786: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36577/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":44}
2026-08-11T14:35:44Z chronyd exiting
I20260811 14:35:44.796670 2461 test_util.cc:175] -----------------------------------------------
I20260811 14:35:44.796779 2461 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458919184199-2461-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (243 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 14:35:44.797729 2461 mini_oidc.cc:152] Starting JWKS server
I20260811 14:35:44.798455 2461 webserver.cc:460] Webserver started at https://127.0.0.1:36705/ using document root <none> and password file <none>
I20260811 14:35:44.798521 2461 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 14:35:44.798746 2461 webserver.cc:460] Webserver started at http://127.0.0.1:39147/ using document root <none> and password file <none>
2026-08-11T14:35:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:44Z Disabled control of system clock
I20260811 14:35:44.808483 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:45221
--webserver_interface=127.2.103.126
--webserver_port=0
--builtin_ntp_servers=127.2.103.84:34013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:45221
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36705/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458919184199-2461-0/testchainca.cert with env {}
W20260811 14:35:44.895996 11167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:44.896219 11167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:44.896243 11167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:44.897370 11167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:44.897408 11167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:44.897421 11167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:44.897434 11167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:44.897446 11167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 14:35:44.897459 11167 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 14:35:44.897476 11167 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36705/jwks.json
I20260811 14:35:44.899073 11167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:34013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:45221
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:45221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36705/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458919184199-2461-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:44.899296 11167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:44.899526 11167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:44.904497 11174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:44.904528 11177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:44.904656 11167 server_base.cc:1034] running on GCE node
W20260811 14:35:44.904526 11180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:44.905022 11167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:44.905320 11167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:44.906484 11167 hybrid_clock.cc:648] HybridClock initialized: now 1786458944906474 us; error 32 us; skew 500 ppm
I20260811 14:35:44.907864 11167 webserver.cc:460] Webserver started at http://127.2.103.126:40539/ using document root <none> and password file <none>
I20260811 14:35:44.908109 11167 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:44.908159 11167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:44.908336 11167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:44.909442 11167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "dea6ef5a17c7426a96f45142d474ff22"
format_stamp: "Formatted at 2026-08-11 14:35:44 on dist-test-slave-twwt"
server_key: "cbefe050509fe96c6e503aebe9bbc2f7"
server_key_iv: "a91a02dc6f140942b8ff9f4fcf144fa0"
server_key_version: "encryptionkey@0"
I20260811 14:35:44.909818 11167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "dea6ef5a17c7426a96f45142d474ff22"
format_stamp: "Formatted at 2026-08-11 14:35:44 on dist-test-slave-twwt"
server_key: "cbefe050509fe96c6e503aebe9bbc2f7"
server_key_iv: "a91a02dc6f140942b8ff9f4fcf144fa0"
server_key_version: "encryptionkey@0"
I20260811 14:35:44.911348 11167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:35:44.912091 11188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:44.912262 11167 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 14:35:44.912369 11167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "dea6ef5a17c7426a96f45142d474ff22"
format_stamp: "Formatted at 2026-08-11 14:35:44 on dist-test-slave-twwt"
server_key: "cbefe050509fe96c6e503aebe9bbc2f7"
server_key_iv: "a91a02dc6f140942b8ff9f4fcf144fa0"
server_key_version: "encryptionkey@0"
I20260811 14:35:44.912441 11167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:44.925284 11167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:44.925607 11167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:44.925760 11167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:44.929936 11167 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:45221
I20260811 14:35:44.929971 11274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:45221 every 8 connection(s)
I20260811 14:35:44.930351 11167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:44.931442 11275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:44.933868 11275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22: Bootstrap starting.
I20260811 14:35:44.933936 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 11167
I20260811 14:35:44.934046 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:35:44.934218 2461 external_mini_cluster.cc:1428] Setting key e1c5ca7a7ab5c346447a10c1c391e8dd
I20260811 14:35:44.934738 11275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:44.935079 11275 log.cc:824] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:44.935874 11275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22: No bootstrap required, opened a new log
I20260811 14:35:44.938001 11275 raft_consensus.cc:348] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea6ef5a17c7426a96f45142d474ff22" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45221 } }
I20260811 14:35:44.938189 11275 raft_consensus.cc:374] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:44.938246 11275 raft_consensus.cc:729] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dea6ef5a17c7426a96f45142d474ff22, State: Initialized, Role: FOLLOWER
I20260811 14:35:44.938387 11275 consensus_queue.cc:260] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea6ef5a17c7426a96f45142d474ff22" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45221 } }
I20260811 14:35:44.938463 11275 raft_consensus.cc:388] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:44.938503 11275 raft_consensus.cc:482] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:44.938555 11275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:44.939283 11275 raft_consensus.cc:504] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea6ef5a17c7426a96f45142d474ff22" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45221 } }
I20260811 14:35:44.939412 11275 leader_election.cc:302] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dea6ef5a17c7426a96f45142d474ff22; no voters:
I20260811 14:35:44.939594 11275 leader_election.cc:288] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:44.939649 11285 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:44.939810 11285 raft_consensus.cc:686] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22 [term 1 LEADER]: Becoming Leader. State: Replica: dea6ef5a17c7426a96f45142d474ff22, State: Running, Role: LEADER
I20260811 14:35:44.939908 11275 sys_catalog.cc:564] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:44.939898 11285 consensus_queue.cc:237] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea6ef5a17c7426a96f45142d474ff22" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45221 } }
I20260811 14:35:44.940315 11285 sys_catalog.cc:455] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dea6ef5a17c7426a96f45142d474ff22. Latest consensus state: current_term: 1 leader_uuid: "dea6ef5a17c7426a96f45142d474ff22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea6ef5a17c7426a96f45142d474ff22" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45221 } } }
I20260811 14:35:44.940428 11285 sys_catalog.cc:458] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:44.940310 11286 sys_catalog.cc:455] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dea6ef5a17c7426a96f45142d474ff22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dea6ef5a17c7426a96f45142d474ff22" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45221 } } }
I20260811 14:35:44.940626 11286 sys_catalog.cc:458] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:44.940922 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.934779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42948 (local address 127.2.103.126:45221)
0811 14:35:44.934970 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:44.934977 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:44.935003 (+ 26us) server_negotiation.cc:407] Connection header received
0811 14:35:44.935652 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:44.935660 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:44.935733 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:44.935857 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:44.936117 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:44.937019 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:44.938219 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:44.938864 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:44.938959 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:44.939841 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:44.939846 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:44.939846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:44.940069 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:44.940120 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:44.940135 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:44.940208 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:44.940594 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 14:35:44.940682 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
I20260811 14:35:44.941138 11295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:44.941627 11295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:44.941919 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:35:44.943470 11295 catalog_manager.cc:1347] Generated new cluster ID: 389b52122ef1481a8250eb69acbae9bb
I20260811 14:35:44.943544 11295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:35:44.967916 11295 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:44.968719 11295 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:44.976011 11295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dea6ef5a17c7426a96f45142d474ff22: Generated new TSK 0
I20260811 14:35:44.976284 11295 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 14:35:45.010488 11318 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 14:35:45.010597 11318 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 14:35:45.010950 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:44.989164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42958 (local address 127.2.103.126:45221)
0811 14:35:44.989240 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:44.989242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:44.989389 (+ 147us) server_negotiation.cc:407] Connection header received
0811 14:35:44.989477 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:44.989478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:44.989489 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:44.989557 (+ 68us) server_negotiation.cc:226] Negotiated authn=JWT
0811 14:35:44.989700 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:44.990208 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:44.991438 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:44.991644 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:44.991725 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:44.991918 (+ 193us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 14:35:45.010658 (+ 18740us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 14:35:45.010892 (+ 234us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42958: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":40}
W20260811 14:35:45.011024 11281 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42958: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 14:35:45.011138 11317 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:44.989094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45221 (local address 127.0.0.1:42958)
0811 14:35:44.989301 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:44.989315 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:44.989395 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:44.989573 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:44.989574 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:44.989576 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 14:35:44.989635 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:44.989637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:44.990320 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:44.990322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:44.991321 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:44.991325 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:44.991733 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:44.991734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:44.991798 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:44.991805 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 14:35:45.010849 (+ 19044us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 14:35:45.011004 (+ 155us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.103.126:45221: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":118,"threads_started":1}
I20260811 14:35:45.011528 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 11167
2026-08-11T14:35:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (261 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 14:35:45.059576 2461 mini_oidc.cc:152] Starting JWKS server
I20260811 14:35:45.060277 2461 webserver.cc:460] Webserver started at https://127.0.0.1:34123/ using document root <none> and password file <none>
I20260811 14:35:45.060323 2461 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 14:35:45.060485 2461 webserver.cc:460] Webserver started at http://127.0.0.1:43841/ using document root <none> and password file <none>
2026-08-11T14:35:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:45Z Disabled control of system clock
I20260811 14:35:45.070047 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:45459
--webserver_interface=127.2.103.126
--webserver_port=0
--builtin_ntp_servers=127.2.103.84:37025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:45459
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34123/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 14:35:45.156145 11333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:45.156471 11333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:45.156515 11333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:45.158290 11333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:45.158350 11333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:45.158363 11333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:45.158377 11333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:45.158389 11333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 14:35:45.158402 11333 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 14:35:45.158413 11333 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34123/jwks.json
I20260811 14:35:45.160007 11333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:37025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:45459
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:45459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34123/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:45.160247 11333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:45.160542 11333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:45.165686 11339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:45.165695 11338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:45.165709 11341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:45.165949 11333 server_base.cc:1034] running on GCE node
I20260811 14:35:45.166214 11333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:45.166565 11333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:45.167749 11333 hybrid_clock.cc:648] HybridClock initialized: now 1786458945167728 us; error 40 us; skew 500 ppm
I20260811 14:35:45.169044 11333 webserver.cc:460] Webserver started at http://127.2.103.126:37421/ using document root <none> and password file <none>
I20260811 14:35:45.169323 11333 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:45.169405 11333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:45.169554 11333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:45.170745 11333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "d881a68eba77410fb807ec7734318d4f"
format_stamp: "Formatted at 2026-08-11 14:35:45 on dist-test-slave-twwt"
server_key: "b8505cf1e0221f18043b2b210be290a2"
server_key_iv: "a518ea9cb4c7acab2643f13d221492db"
server_key_version: "encryptionkey@0"
I20260811 14:35:45.171119 11333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "d881a68eba77410fb807ec7734318d4f"
format_stamp: "Formatted at 2026-08-11 14:35:45 on dist-test-slave-twwt"
server_key: "b8505cf1e0221f18043b2b210be290a2"
server_key_iv: "a518ea9cb4c7acab2643f13d221492db"
server_key_version: "encryptionkey@0"
I20260811 14:35:45.172654 11333 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 14:35:45.173499 11355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:45.173694 11333 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:45.173802 11333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "d881a68eba77410fb807ec7734318d4f"
format_stamp: "Formatted at 2026-08-11 14:35:45 on dist-test-slave-twwt"
server_key: "b8505cf1e0221f18043b2b210be290a2"
server_key_iv: "a518ea9cb4c7acab2643f13d221492db"
server_key_version: "encryptionkey@0"
I20260811 14:35:45.173877 11333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:45.193092 11333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:45.193477 11333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:45.193645 11333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:45.197983 11333 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:45459
I20260811 14:35:45.198022 11438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:45459 every 8 connection(s)
I20260811 14:35:45.198410 11333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:45.199338 11439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:45.202451 11439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f: Bootstrap starting.
I20260811 14:35:45.203163 11439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:45.203481 11439 log.cc:824] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:45.205090 11439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f: No bootstrap required, opened a new log
I20260811 14:35:45.205811 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 11333
I20260811 14:35:45.205902 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:35:45.206087 2461 external_mini_cluster.cc:1428] Setting key 927a76dbca0835322e11010b21c8ba88
I20260811 14:35:45.207211 11439 raft_consensus.cc:348] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d881a68eba77410fb807ec7734318d4f" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45459 } }
I20260811 14:35:45.207386 11439 raft_consensus.cc:374] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:45.207427 11439 raft_consensus.cc:729] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d881a68eba77410fb807ec7734318d4f, State: Initialized, Role: FOLLOWER
I20260811 14:35:45.207568 11439 consensus_queue.cc:260] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d881a68eba77410fb807ec7734318d4f" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45459 } }
I20260811 14:35:45.207671 11439 raft_consensus.cc:388] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:45.207723 11439 raft_consensus.cc:482] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:45.207780 11439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:45.208400 11439 raft_consensus.cc:504] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d881a68eba77410fb807ec7734318d4f" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45459 } }
I20260811 14:35:45.208515 11439 leader_election.cc:302] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d881a68eba77410fb807ec7734318d4f; no voters:
I20260811 14:35:45.208730 11439 leader_election.cc:288] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:45.208746 11446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:45.208885 11446 raft_consensus.cc:686] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f [term 1 LEADER]: Becoming Leader. State: Replica: d881a68eba77410fb807ec7734318d4f, State: Running, Role: LEADER
I20260811 14:35:45.208977 11446 consensus_queue.cc:237] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d881a68eba77410fb807ec7734318d4f" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45459 } }
I20260811 14:35:45.209052 11439 sys_catalog.cc:564] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:45.209322 11447 sys_catalog.cc:455] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d881a68eba77410fb807ec7734318d4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d881a68eba77410fb807ec7734318d4f" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45459 } } }
I20260811 14:35:45.209563 11447 sys_catalog.cc:458] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:45.209492 11449 sys_catalog.cc:455] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d881a68eba77410fb807ec7734318d4f. Latest consensus state: current_term: 1 leader_uuid: "d881a68eba77410fb807ec7734318d4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d881a68eba77410fb807ec7734318d4f" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45459 } } }
I20260811 14:35:45.209841 11449 sys_catalog.cc:458] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:45.210222 11458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:45.210779 11458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:45.212869 11444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:45.206662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38920 (local address 127.2.103.126:45459)
0811 14:35:45.206878 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:45.206885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:45.206910 (+ 25us) server_negotiation.cc:407] Connection header received
0811 14:35:45.207177 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:45.207184 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:45.207262 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:45.207393 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:45.207659 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:45.209263 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:45.210557 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:45.211179 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:45.211318 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:45.212099 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:45.212104 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:45.212104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:45.212330 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:45.212375 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:45.212390 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:45.212458 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:45.212554 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 14:35:45.212639 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
I20260811 14:35:45.213970 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:35:45.215083 11458 catalog_manager.cc:1347] Generated new cluster ID: 33a08ceb47f142a1a35e7e3758b20125
I20260811 14:35:45.215140 11458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:35:45.236598 11458 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:45.237362 11458 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:45.242748 11458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d881a68eba77410fb807ec7734318d4f: Generated new TSK 0
I20260811 14:35:45.242981 11458 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:35:45.257004 11444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:45.253099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38926 (local address 127.2.103.126:45459)
0811 14:35:45.253162 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:45.253165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:45.253257 (+ 92us) server_negotiation.cc:407] Connection header received
0811 14:35:45.253324 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:45.253326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:45.253340 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:45.253432 (+ 92us) server_negotiation.cc:226] Negotiated authn=JWT
0811 14:35:45.253563 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:45.254357 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:45.255186 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:45.255504 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:45.255581 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:45.255731 (+ 150us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 14:35:45.256770 (+ 1039us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34123/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 14:35:45.256916 (+ 146us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38926: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34123/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":35}
W20260811 14:35:45.257076 11444 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38926: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34123/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 14:35:45.259078 11486 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:45.253002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45459 (local address 127.0.0.1:38926)
0811 14:35:45.253190 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:45.253200 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:45.253259 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:45.253438 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:45.253439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:45.253441 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 14:35:45.253499 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:45.253501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:45.254439 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:45.254440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:45.255102 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:45.255104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:45.255590 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:45.255591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:45.255633 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:45.255639 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 14:35:45.256888 (+ 1249us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34123/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 14:35:45.258969 (+ 2081us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.103.126:45459: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34123/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":1851,"thread_start_us":119,"threads_started":1}
I20260811 14:35:45.259773 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 11333
E20260811 14:35:45.273844 11487 webserver.cc:533] Webserver: sslize failed:
E20260811 14:35:45.273932 11487 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T14:35:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (246 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T14:35:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:45Z Disabled control of system clock
I20260811 14:35:45.315868 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:38411
--webserver_interface=127.2.103.126
--webserver_port=0
--builtin_ntp_servers=127.2.103.84:36649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:38411
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458919184199-2461-0/testchainca.cert with env {}
W20260811 14:35:45.399982 11494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:45.400210 11494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:45.400233 11494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:45.401433 11494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:45.401484 11494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:45.401499 11494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:45.401511 11494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:45.401525 11494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 14:35:45.401537 11494 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 14:35:45.401587 11494 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 14:35:45.403203 11494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:36649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:38411
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:38411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458919184199-2461-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:45.403475 11494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:45.403708 11494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:45.408932 11507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:45.408895 11504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:45.409157 11494 server_base.cc:1034] running on GCE node
W20260811 14:35:45.408919 11510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:45.410342 11494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:45.410724 11494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:45.411875 11494 hybrid_clock.cc:648] HybridClock initialized: now 1786458945411866 us; error 33 us; skew 500 ppm
I20260811 14:35:45.413241 11494 webserver.cc:460] Webserver started at http://127.2.103.126:46739/ using document root <none> and password file <none>
I20260811 14:35:45.413522 11494 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:45.413586 11494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:45.413739 11494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:45.414947 11494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "7a1d10c5e7cc41a9abaf0ef21ba458cc"
format_stamp: "Formatted at 2026-08-11 14:35:45 on dist-test-slave-twwt"
server_key: "3c784d5f9b646249b34338fcd1101c1b"
server_key_iv: "c37384149dbf93b3641f366ba5aeea07"
server_key_version: "encryptionkey@0"
I20260811 14:35:45.415390 11494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "7a1d10c5e7cc41a9abaf0ef21ba458cc"
format_stamp: "Formatted at 2026-08-11 14:35:45 on dist-test-slave-twwt"
server_key: "3c784d5f9b646249b34338fcd1101c1b"
server_key_iv: "c37384149dbf93b3641f366ba5aeea07"
server_key_version: "encryptionkey@0"
I20260811 14:35:45.417064 11494 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:35:45.417936 11516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:45.418145 11494 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:45.418257 11494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "7a1d10c5e7cc41a9abaf0ef21ba458cc"
format_stamp: "Formatted at 2026-08-11 14:35:45 on dist-test-slave-twwt"
server_key: "3c784d5f9b646249b34338fcd1101c1b"
server_key_iv: "c37384149dbf93b3641f366ba5aeea07"
server_key_version: "encryptionkey@0"
I20260811 14:35:45.418334 11494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:45.433877 11494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:45.434212 11494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:45.434376 11494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:45.438632 11494 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:38411
I20260811 14:35:45.438774 11601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:38411 every 8 connection(s)
I20260811 14:35:45.439100 11494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:45.440186 11605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:45.440953 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 11494
I20260811 14:35:45.441076 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:35:45.441267 2461 external_mini_cluster.cc:1428] Setting key 16526775b14e4863996912d6fb3a3631
I20260811 14:35:45.443451 11605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc: Bootstrap starting.
I20260811 14:35:45.444175 11605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:45.444519 11605 log.cc:824] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:45.445326 11605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc: No bootstrap required, opened a new log
I20260811 14:35:45.447072 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:45.441858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37712 (local address 127.2.103.126:38411)
0811 14:35:45.442067 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:45.442075 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:45.442103 (+ 28us) server_negotiation.cc:407] Connection header received
0811 14:35:45.442535 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:45.442542 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:45.442654 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:45.442780 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:45.443029 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:45.443975 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:45.445065 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:45.445519 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:45.445614 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:45.446344 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:45.446349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:45.446349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:45.446521 (+ 172us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:45.446556 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:45.446567 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:45.446679 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:45.446764 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 14:35:45.446850 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":95,"threads_started":1}
I20260811 14:35:45.447834 11605 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a1d10c5e7cc41a9abaf0ef21ba458cc" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38411 } }
I20260811 14:35:45.448020 11605 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:45.448057 11605 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a1d10c5e7cc41a9abaf0ef21ba458cc, State: Initialized, Role: FOLLOWER
I20260811 14:35:45.448194 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:35:45.448185 11605 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a1d10c5e7cc41a9abaf0ef21ba458cc" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38411 } }
I20260811 14:35:45.448282 11605 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:45.448323 11605 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:45.448369 11605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:45.449178 11605 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a1d10c5e7cc41a9abaf0ef21ba458cc" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38411 } }
I20260811 14:35:45.449308 11605 leader_election.cc:302] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a1d10c5e7cc41a9abaf0ef21ba458cc; no voters:
I20260811 14:35:45.449499 11605 leader_election.cc:288] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:45.449559 11615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:45.449725 11615 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc [term 1 LEADER]: Becoming Leader. State: Replica: 7a1d10c5e7cc41a9abaf0ef21ba458cc, State: Running, Role: LEADER
I20260811 14:35:45.449779 11605 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:45.449822 11615 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a1d10c5e7cc41a9abaf0ef21ba458cc" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38411 } }
I20260811 14:35:45.450214 11617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a1d10c5e7cc41a9abaf0ef21ba458cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a1d10c5e7cc41a9abaf0ef21ba458cc" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38411 } } }
I20260811 14:35:45.450331 11617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:45.450214 11619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a1d10c5e7cc41a9abaf0ef21ba458cc. Latest consensus state: current_term: 1 leader_uuid: "7a1d10c5e7cc41a9abaf0ef21ba458cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a1d10c5e7cc41a9abaf0ef21ba458cc" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38411 } } }
I20260811 14:35:45.450385 11619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:45.451107 11627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:45.451692 11627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:45.453341 11627 catalog_manager.cc:1347] Generated new cluster ID: 43ca4bdeef304f9bbe7e6152b1d26143
I20260811 14:35:45.453414 11627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:35:45.464725 11627 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:45.465310 11627 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:45.473960 11627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a1d10c5e7cc41a9abaf0ef21ba458cc: Generated new TSK 0
I20260811 14:35:45.474222 11627 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:35:45.595356 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:45.482418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37720 (local address 127.2.103.126:38411)
0811 14:35:45.482483 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:45.482486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:45.482586 (+ 100us) server_negotiation.cc:407] Connection header received
0811 14:35:45.482704 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:45.482705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:45.482728 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:45.482817 (+ 89us) server_negotiation.cc:226] Negotiated authn=JWT
0811 14:35:45.482941 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:45.483751 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:45.484552 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:45.484895 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:45.484968 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:45.485139 (+ 171us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 14:35:45.595095 (+109956us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 14:35:45.595279 (+ 184us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37720: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":37}
W20260811 14:35:45.595484 11608 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37720: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 14:35:45.595496 11652 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:45.482321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:38411 (local address 127.0.0.1:37720)
0811 14:35:45.482513 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:45.482523 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:45.482587 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:45.482815 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:45.482816 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:45.482817 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0811 14:35:45.482874 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:45.482876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:45.483855 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:45.483856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:45.484472 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:45.484474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:45.484984 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:45.484985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:45.485029 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:45.485035 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 14:35:45.595262 (+110227us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 14:35:45.595411 (+ 149us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.103.126:38411: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":119,"threads_started":1}
I20260811 14:35:45.595921 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 11494
2026-08-11T14:35:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (323 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (97 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (98 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:45 dist-test-slave-twwt krb5kdc[11668](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:45 dist-test-slave-twwt krb5kdc[11668](info): set up 2 sockets
Aug 11 14:35:45 dist-test-slave-twwt krb5kdc[11668](info): commencing operation
krb5kdc: starting...
W20260811 14:35:47.128021 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:35:47 dist-test-slave-twwt krb5kdc[11668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:35:47.266104 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:38207
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:38207
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:47.354336 11686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:47.354553 11686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:47.354576 11686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:47.355789 11686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:47.355841 11686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:47.355856 11686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:47.355897 11686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:47.355926 11686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:47.357396 11686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:38207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:38207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:47.357656 11686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:47.357892 11686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:47.363289 11699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:47.363309 11697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:47.363406 11686 server_base.cc:1034] running on GCE node
W20260811 14:35:47.363507 11701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:47.363780 11686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:47.364106 11686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:47.365285 11686 hybrid_clock.cc:648] HybridClock initialized: now 1786458947365276 us; error 36 us; skew 500 ppm
Aug 11 14:35:47 dist-test-slave-twwt krb5kdc[11668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458947, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:47.368008 11686 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:35:47.368479 11686 webserver.cc:460] Webserver started at http://127.2.103.126:45023/ using document root <none> and password file <none>
I20260811 14:35:47.368669 11686 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:47.368709 11686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:47.368800 11686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:47.369853 11686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "bb97884b914846908db5b2213baba14f"
format_stamp: "Formatted at 2026-08-11 14:35:47 on dist-test-slave-twwt"
server_key: "259c47ecf4870387dbe5983a1df9059c"
server_key_iv: "9df6ae1d484c554abae3dbebda69f989"
server_key_version: "encryptionkey@0"
I20260811 14:35:47.370167 11686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "bb97884b914846908db5b2213baba14f"
format_stamp: "Formatted at 2026-08-11 14:35:47 on dist-test-slave-twwt"
server_key: "259c47ecf4870387dbe5983a1df9059c"
server_key_iv: "9df6ae1d484c554abae3dbebda69f989"
server_key_version: "encryptionkey@0"
I20260811 14:35:47.371585 11686 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:35:47.372407 11711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:47.372587 11686 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 14:35:47.372685 11686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "bb97884b914846908db5b2213baba14f"
format_stamp: "Formatted at 2026-08-11 14:35:47 on dist-test-slave-twwt"
server_key: "259c47ecf4870387dbe5983a1df9059c"
server_key_iv: "9df6ae1d484c554abae3dbebda69f989"
server_key_version: "encryptionkey@0"
I20260811 14:35:47.372766 11686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:47.380168 11686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:47.382323 11686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:47.382498 11686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:47.386482 11686 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:38207
I20260811 14:35:47.386518 11806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:38207 every 8 connection(s)
I20260811 14:35:47.386963 11686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:47.387964 11808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:47.390376 11808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f: Bootstrap starting.
I20260811 14:35:47.391048 11808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:47.391342 11808 log.cc:824] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:47.392093 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 11686
I20260811 14:35:47.392197 11808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f: No bootstrap required, opened a new log
I20260811 14:35:47.392220 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:35:47.392431 2461 external_mini_cluster.cc:1428] Setting key 0fb66dc6dead29adf1cfb21037d32fb6
I20260811 14:35:47.394124 11808 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb97884b914846908db5b2213baba14f" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38207 } }
I20260811 14:35:47.394294 11808 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:47.394363 11808 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb97884b914846908db5b2213baba14f, State: Initialized, Role: FOLLOWER
I20260811 14:35:47.394532 11808 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb97884b914846908db5b2213baba14f" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38207 } }
I20260811 14:35:47.394649 11808 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:47.394690 11808 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:47.394744 11808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:47.395747 11808 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb97884b914846908db5b2213baba14f" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38207 } }
I20260811 14:35:47.395872 11808 leader_election.cc:302] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb97884b914846908db5b2213baba14f; no voters:
I20260811 14:35:47.396060 11808 leader_election.cc:288] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:47.396085 11813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:47.396265 11813 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f [term 1 LEADER]: Becoming Leader. State: Replica: bb97884b914846908db5b2213baba14f, State: Running, Role: LEADER
I20260811 14:35:47.396404 11808 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:47.396418 11813 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb97884b914846908db5b2213baba14f" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38207 } }
I20260811 14:35:47.397022 11814 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb97884b914846908db5b2213baba14f. Latest consensus state: current_term: 1 leader_uuid: "bb97884b914846908db5b2213baba14f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb97884b914846908db5b2213baba14f" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38207 } } }
I20260811 14:35:47.397167 11814 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:47.397365 11815 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb97884b914846908db5b2213baba14f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb97884b914846908db5b2213baba14f" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38207 } } }
I20260811 14:35:47.397485 11815 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:47.397888 11829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:47.398561 11829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 14:35:47 dist-test-slave-twwt krb5kdc[11668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:47.400943 11829 catalog_manager.cc:1347] Generated new cluster ID: 2ea935b959734c51806e91e6fc9a9e6b
I20260811 14:35:47.400998 11829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:35:47.404374 11812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:47.392985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53456 (local address 127.2.103.126:38207)
0811 14:35:47.393222 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:47.393230 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:47.393261 (+ 31us) server_negotiation.cc:407] Connection header received
0811 14:35:47.394101 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:47.394108 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:47.394157 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:47.394296 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:47.395127 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:47.396136 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:47.397153 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:47.397808 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:47.397907 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:47.401008 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:47.401012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:47.401012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:47.401030 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:47.402808 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:47.403169 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:47.403173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:47.403173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:47.403285 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:47.403497 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:47.403498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:47.403499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:47.403739 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:47.403911 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:47.404013 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 14:35:47.404164 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":119,"threads_started":1}
I20260811 14:35:47.440414 11829 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:47.441319 11829 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:47.448935 11829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb97884b914846908db5b2213baba14f: Generated new TSK 0
I20260811 14:35:47.449190 11829 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
I20260811 14:35:47.486991 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38207
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:47.574388 11847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:47.574653 11847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:47.574702 11847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:47.576026 11847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:47.576090 11847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:47.576139 11847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:35:47.577852 11847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:47.578157 11847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:47.578421 11847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:47.583582 11858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:47.583772 11860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:47.583957 11856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:47.584252 11847 server_base.cc:1034] running on GCE node
I20260811 14:35:47.584481 11847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:47.584753 11847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:47.585918 11847 hybrid_clock.cc:648] HybridClock initialized: now 1786458947585903 us; error 33 us; skew 500 ppm
Aug 11 14:35:47 dist-test-slave-twwt krb5kdc[11668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458947, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:47.588601 11847 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:35:47.589246 11847 webserver.cc:460] Webserver started at http://127.2.103.65:44285/ using document root <none> and password file <none>
I20260811 14:35:47.589556 11847 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:47.589635 11847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:47.589805 11847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:47.591084 11847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "db2b40e3a36d4438b4ee161c2729fc81"
format_stamp: "Formatted at 2026-08-11 14:35:47 on dist-test-slave-twwt"
server_key: "7fceba8418355a8205bcddbfa907879e"
server_key_iv: "e214182fb6fe1f45c6e4be614c410b70"
server_key_version: "encryptionkey@0"
I20260811 14:35:47.591499 11847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "db2b40e3a36d4438b4ee161c2729fc81"
format_stamp: "Formatted at 2026-08-11 14:35:47 on dist-test-slave-twwt"
server_key: "7fceba8418355a8205bcddbfa907879e"
server_key_iv: "e214182fb6fe1f45c6e4be614c410b70"
server_key_version: "encryptionkey@0"
I20260811 14:35:47.593178 11847 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:35:47.594027 11873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:47.594197 11847 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:47.594318 11847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "db2b40e3a36d4438b4ee161c2729fc81"
format_stamp: "Formatted at 2026-08-11 14:35:47 on dist-test-slave-twwt"
server_key: "7fceba8418355a8205bcddbfa907879e"
server_key_iv: "e214182fb6fe1f45c6e4be614c410b70"
server_key_version: "encryptionkey@0"
I20260811 14:35:47.594415 11847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:47.631394 11847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:47.634222 11847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:47.634420 11847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:47.634783 11847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:47.635231 11847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:47.635293 11847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:47.635331 11847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:47.635356 11847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:47.643085 11847 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:44267
I20260811 14:35:47.643136 12074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:44267 every 8 connection(s)
I20260811 14:35:47.643537 11847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:35:47 dist-test-slave-twwt krb5kdc[11668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458947, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:47.653519 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 11847
I20260811 14:35:47.653642 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:35:47.653874 2461 external_mini_cluster.cc:1428] Setting key 55e490ae321f70a82f96f795832dadb4
I20260811 14:35:47.655198 11812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:47.644398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:48573 (local address 127.2.103.126:38207)
0811 14:35:47.644457 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:47.644460 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:47.644707 (+ 247us) server_negotiation.cc:407] Connection header received
0811 14:35:47.645765 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:47.645767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:47.645784 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:47.645892 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:47.646574 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:47.647329 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:47.648432 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:47.648821 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:47.648905 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:47.651290 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:47.651292 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:47.651292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:47.651307 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:47.653157 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:47.653795 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:47.653797 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:47.653797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:47.653870 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:47.654280 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:47.654281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:47.654281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:47.654422 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:47.654496 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:47.654952 (+ 456us) server_negotiation.cc:299] Negotiation successful
0811 14:35:47.655062 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 14:35:47.655476 12081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:47.644362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:38207 (local address 127.2.103.65:48573)
0811 14:35:47.644553 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:47.644601 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:47.645651 (+ 1050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:47.645964 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:47.645967 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:47.646261 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:47.646472 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:47.646476 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:47.647418 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:47.647419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:47.648336 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:47.648339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:47.648930 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:47.648932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:47.649055 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:47.649673 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:47.649681 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:47.651172 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:47.653251 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:47.653257 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:47.653259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:47.653665 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:47.653984 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:47.653985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:47.653986 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:47.654133 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:47.654503 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:47.654512 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:47.654763 (+ 251us) client_negotiation.cc:769] Sending connection context
0811 14:35:47.654915 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 14:35:47.655139 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":101,"threads_started":1}
I20260811 14:35:47.656088 12076 heartbeater.cc:347] Connected to a master server at 127.2.103.126:38207
I20260811 14:35:47.656206 12076 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:47.656478 12076 heartbeater.cc:511] Master 127.2.103.126:38207 requested a full tablet report, sending...
I20260811 14:35:47.657146 11740 ts_manager.cc:194] Registered new tserver with Master: db2b40e3a36d4438b4ee161c2729fc81 (127.2.103.65:44267)
I20260811 14:35:47.658303 11740 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:48573
WARNING: no policy specified for kudu/127.2.103.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
I20260811 14:35:47.690493 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38207
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:47.783593 12084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:47.783810 12084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:47.783835 12084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:47.784955 12084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:47.784989 12084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:47.785017 12084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:35:47.786549 12084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:47.786793 12084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:47.786995 12084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:47.792187 12093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:47.792245 12091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:47.792222 12084 server_base.cc:1034] running on GCE node
W20260811 14:35:47.792372 12089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:47.792603 12084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:47.792904 12084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:47.794045 12084 hybrid_clock.cc:648] HybridClock initialized: now 1786458947794024 us; error 36 us; skew 500 ppm
Aug 11 14:35:47 dist-test-slave-twwt krb5kdc[11668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458947, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:47.796619 12084 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:35:47.797130 12084 webserver.cc:460] Webserver started at http://127.2.103.66:40575/ using document root <none> and password file <none>
I20260811 14:35:47.797436 12084 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:47.797505 12084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:47.797668 12084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:47.798893 12084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "58f3e7d626df4c27827ebc567073f3b0"
format_stamp: "Formatted at 2026-08-11 14:35:47 on dist-test-slave-twwt"
server_key: "471c205697e08bf95a098822bcf2d255"
server_key_iv: "18ecfb1552875099cacbe38a2f8ffd87"
server_key_version: "encryptionkey@0"
I20260811 14:35:47.799263 12084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "58f3e7d626df4c27827ebc567073f3b0"
format_stamp: "Formatted at 2026-08-11 14:35:47 on dist-test-slave-twwt"
server_key: "471c205697e08bf95a098822bcf2d255"
server_key_iv: "18ecfb1552875099cacbe38a2f8ffd87"
server_key_version: "encryptionkey@0"
I20260811 14:35:47.800561 12084 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:47.801296 12105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:47.801466 12084 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:47.801550 12084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "58f3e7d626df4c27827ebc567073f3b0"
format_stamp: "Formatted at 2026-08-11 14:35:47 on dist-test-slave-twwt"
server_key: "471c205697e08bf95a098822bcf2d255"
server_key_iv: "18ecfb1552875099cacbe38a2f8ffd87"
server_key_version: "encryptionkey@0"
I20260811 14:35:47.801607 12084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:47.829178 12084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:47.831207 12084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:47.831326 12084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:47.831539 12084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:47.831915 12084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:47.831954 12084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:47.831976 12084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:47.832023 12084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:47.838012 12084 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:34755
I20260811 14:35:47.838116 12304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:34755 every 8 connection(s)
I20260811 14:35:47.838438 12084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
Aug 11 14:35:47 dist-test-slave-twwt krb5kdc[11668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458947, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:47.847357 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 12084
I20260811 14:35:47.847496 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:35:47.847714 2461 external_mini_cluster.cc:1428] Setting key 6d360a7cbdcaa1d37023a20896d8f87f
I20260811 14:35:47.849174 11812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:47.839300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:40163 (local address 127.2.103.126:38207)
0811 14:35:47.839380 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:47.839382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:47.839568 (+ 186us) server_negotiation.cc:407] Connection header received
0811 14:35:47.840580 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:47.840582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:47.840594 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:47.840662 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:47.841342 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:47.841844 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:47.842950 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:47.843158 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:47.843218 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:47.846033 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:47.846035 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:47.846035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:47.846043 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:47.847548 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:47.848085 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:47.848086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:47.848087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:47.848173 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:47.848482 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:47.848484 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:47.848484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:47.848618 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:47.848692 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:47.848990 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 14:35:47.849083 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 14:35:47.849396 12311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:47.839250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:38207 (local address 127.2.103.66:40163)
0811 14:35:47.839429 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:47.839474 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:47.840456 (+ 982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:47.840759 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:47.840763 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:47.841052 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:47.841259 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:47.841262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:47.841953 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:47.841954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:47.842847 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:47.842850 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:47.843252 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:47.843253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:47.843383 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:47.843925 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:47.843935 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:47.845938 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:47.847628 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:47.847640 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:47.847642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:47.847987 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:47.848241 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:47.848242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:47.848242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:47.848390 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:47.848694 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:47.848702 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:47.848844 (+ 142us) client_negotiation.cc:769] Sending connection context
0811 14:35:47.848957 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 14:35:47.849130 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":95,"threads_started":1}
I20260811 14:35:47.849926 12307 heartbeater.cc:347] Connected to a master server at 127.2.103.126:38207
I20260811 14:35:47.850047 12307 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:47.850266 12307 heartbeater.cc:511] Master 127.2.103.126:38207 requested a full tablet report, sending...
I20260811 14:35:47.850798 11740 ts_manager.cc:194] Registered new tserver with Master: 58f3e7d626df4c27827ebc567073f3b0 (127.2.103.66:34755)
I20260811 14:35:47.851199 11740 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:40163
WARNING: no policy specified for kudu/127.2.103.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
I20260811 14:35:47.877869 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38207
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:47.967475 12316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:47.967712 12316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:47.967753 12316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:47.968926 12316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:47.968988 12316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:47.969035 12316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:35:47.970575 12316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:47.970897 12316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:47.971146 12316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:47.976673 12330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:47.976689 12327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:47.976676 12328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:47.976694 12316 server_base.cc:1034] running on GCE node
I20260811 14:35:47.977270 12316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:47.977548 12316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:47.978700 12316 hybrid_clock.cc:648] HybridClock initialized: now 1786458947978683 us; error 33 us; skew 500 ppm
Aug 11 14:35:47 dist-test-slave-twwt krb5kdc[11668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458947, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:47.981500 12316 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:35:47.981968 12316 webserver.cc:460] Webserver started at http://127.2.103.67:33117/ using document root <none> and password file <none>
I20260811 14:35:47.982232 12316 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:47.982280 12316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:47.982450 12316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:47.983640 12316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "2b23316e3c6c4b4f854b272391efafdb"
format_stamp: "Formatted at 2026-08-11 14:35:47 on dist-test-slave-twwt"
server_key: "a12f2e5e46d450a47e874d7e5f80e680"
server_key_iv: "cbc58739053e849fac8dfd6cdf0b8198"
server_key_version: "encryptionkey@0"
I20260811 14:35:47.984018 12316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "2b23316e3c6c4b4f854b272391efafdb"
format_stamp: "Formatted at 2026-08-11 14:35:47 on dist-test-slave-twwt"
server_key: "a12f2e5e46d450a47e874d7e5f80e680"
server_key_iv: "cbc58739053e849fac8dfd6cdf0b8198"
server_key_version: "encryptionkey@0"
I20260811 14:35:47.985473 12316 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:47.986294 12340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:47.986487 12316 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:47.986594 12316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "2b23316e3c6c4b4f854b272391efafdb"
format_stamp: "Formatted at 2026-08-11 14:35:47 on dist-test-slave-twwt"
server_key: "a12f2e5e46d450a47e874d7e5f80e680"
server_key_iv: "cbc58739053e849fac8dfd6cdf0b8198"
server_key_version: "encryptionkey@0"
I20260811 14:35:47.986724 12316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:48.016255 12316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:48.018258 12316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:48.018437 12316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:48.018761 12316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:48.019104 12316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:48.019157 12316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:48.019201 12316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:48.019229 12316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:48.025487 12316 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:43985
I20260811 14:35:48.025535 12540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:43985 every 8 connection(s)
I20260811 14:35:48.025911 12316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
Aug 11 14:35:48 dist-test-slave-twwt krb5kdc[11668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458947, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:35:48.034332 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 12316
I20260811 14:35:48.034433 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:35:48.034679 2461 external_mini_cluster.cc:1428] Setting key 8b0504746cfe7a8e54ad675475aaccaa
I20260811 14:35:48.035810 11812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:48.026713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:54587 (local address 127.2.103.126:38207)
0811 14:35:48.026802 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:48.026805 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:48.026981 (+ 176us) server_negotiation.cc:407] Connection header received
0811 14:35:48.027931 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:48.027932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:48.027943 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:48.027996 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:48.028591 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:48.029095 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:48.030091 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:48.030303 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:48.030371 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:48.032551 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:48.032553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:48.032553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:48.032560 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:48.034063 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:48.034660 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:48.034662 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:48.034662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:48.034744 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:48.035042 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:48.035043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:48.035043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:48.035213 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:48.035285 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:48.035609 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 14:35:48.035704 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 14:35:48.036005 12546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:48.026670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:38207 (local address 127.2.103.67:54587)
0811 14:35:48.026852 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:48.026893 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:48.027832 (+ 939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:48.028055 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:48.028058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:48.028314 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:48.028523 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:48.028527 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:48.029172 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:48.029173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:48.029990 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:48.029992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:48.030397 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:48.030398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:48.030500 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:48.031021 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:48.031031 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:48.032453 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:48.034151 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:48.034158 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:48.034161 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:48.034492 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:48.034890 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:48.034891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:48.034891 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:48.034995 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:48.035284 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:48.035292 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:48.035474 (+ 182us) client_negotiation.cc:769] Sending connection context
0811 14:35:48.035566 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 14:35:48.035764 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":102,"threads_started":1}
I20260811 14:35:48.036482 12541 heartbeater.cc:347] Connected to a master server at 127.2.103.126:38207
I20260811 14:35:48.036578 12541 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:48.036738 12541 heartbeater.cc:511] Master 127.2.103.126:38207 requested a full tablet report, sending...
I20260811 14:35:48.037102 11740 ts_manager.cc:194] Registered new tserver with Master: 2b23316e3c6c4b4f854b272391efafdb (127.2.103.67:43985)
I20260811 14:35:48.037437 11740 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:54587
I20260811 14:35:48.046650 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/test-admin.keytab.
Aug 11 14:35:48 dist-test-slave-twwt krb5kdc[11668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:48.063735 2461 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 14:35:48.065475 12553 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:48.064705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47748 (local address 127.2.103.65:44267)
0811 14:35:48.064912 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:48.064917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:48.064927 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:35:48.064955 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:48.064957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:48.064971 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:48.065043 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:48.065396 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47748: BlockingRecv error: recv got EOF from 127.0.0.1:47748 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":109,"threads_started":1}
W20260811 14:35:48.065670 2461 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/krb5cc' of type FILE
Aug 11 14:35:48 dist-test-slave-twwt krb5kdc[11668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:48.066708 2461 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 14:35:48 dist-test-slave-twwt krb5kdc[11668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:48.067584 2461 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 14:35:48 dist-test-slave-twwt krb5kdc[11668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.65@KRBTEST.COM
I20260811 14:35:48.075073 12553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:48.068123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47754 (local address 127.2.103.65:44267)
0811 14:35:48.068162 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:48.068164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:48.068190 (+ 26us) server_negotiation.cc:407] Connection header received
0811 14:35:48.068251 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:48.068252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:48.068261 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:48.068309 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:48.068675 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:48.069405 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:48.070095 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:48.070405 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:48.070459 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:48.072511 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:48.072518 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:48.072519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:48.072525 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:48.073978 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:48.074293 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:48.074295 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:48.074295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:48.074348 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:48.074507 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:48.074508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:48.074508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:48.074682 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:48.074793 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:48.074862 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 14:35:48.074942 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260811 14:35:48.077067 12553 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:48.076359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47768 (local address 127.2.103.65:44267)
0811 14:35:48.076395 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:48.076397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:48.076477 (+ 80us) server_negotiation.cc:407] Connection header received
0811 14:35:48.076536 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:48.076536 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:48.076547 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:48.076588 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:48.077007 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47768: BlockingRecv error: recv got EOF from 127.0.0.1:47768 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260811 14:35:48.077142 2461 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/krb5cc' of type FILE
Aug 11 14:35:48 dist-test-slave-twwt krb5kdc[11668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:48.078118 2461 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 14:35:48 dist-test-slave-twwt krb5kdc[11668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:48.079078 2461 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 14:35:48 dist-test-slave-twwt krb5kdc[11668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.65@KRBTEST.COM
I20260811 14:35:48.086531 12553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:48.079660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47776 (local address 127.2.103.65:44267)
0811 14:35:48.079729 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:48.079731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:48.079804 (+ 73us) server_negotiation.cc:407] Connection header received
0811 14:35:48.079824 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:48.079825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:48.079834 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:48.079878 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:48.080303 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:48.080886 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:48.081544 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:48.081813 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:48.081875 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:48.084009 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:48.084011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:48.084011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:48.084018 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:48.085554 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:48.085850 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:48.085851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:48.085851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:48.085903 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:48.086096 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:48.086097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:48.086097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:48.086208 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:48.086297 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:48.086392 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 14:35:48.086462 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 14:35:48.088191 12553 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:48.087651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47784 (local address 127.2.103.65:44267)
0811 14:35:48.087709 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:48.087711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:48.087738 (+ 27us) server_negotiation.cc:407] Connection header received
0811 14:35:48.087811 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:48.087812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:48.087821 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:48.087872 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:48.088141 (+ 269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47784: BlockingRecv error: recv got EOF from 127.0.0.1:47784 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260811 14:35:48.088363 2461 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458919184199-2461-0/krb5kdc/krb5cc' of type FILE
Aug 11 14:35:48 dist-test-slave-twwt krb5kdc[11668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:48.089445 2461 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 14:35:48 dist-test-slave-twwt krb5kdc[11668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:48.090337 2461 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 14:35:48 dist-test-slave-twwt krb5kdc[11668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.65@KRBTEST.COM
I20260811 14:35:48.097648 12553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:48.090886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47786 (local address 127.2.103.65:44267)
0811 14:35:48.090939 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:48.090941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:48.091042 (+ 101us) server_negotiation.cc:407] Connection header received
0811 14:35:48.091092 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:48.091092 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:48.091100 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:48.091145 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:48.091505 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:48.091948 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:48.092658 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:48.092858 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:48.092921 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:48.095084 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:48.095086 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:48.095086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:48.095092 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:48.096308 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:48.096645 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:48.096646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:48.096647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:48.096696 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:48.096935 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:48.096936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:48.096937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:48.097125 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:48.097197 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:48.097443 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 14:35:48.097507 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 14:35:48.098321 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 11847
I20260811 14:35:48.116307 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 12084
I20260811 14:35:48.127003 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 12316
I20260811 14:35:48.148111 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 11686
2026-08-11T14:35:48Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2349 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:48 dist-test-slave-twwt krb5kdc[12573](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:48 dist-test-slave-twwt krb5kdc[12573](info): set up 2 sockets
Aug 11 14:35:48 dist-test-slave-twwt krb5kdc[12573](info): commencing operation
krb5kdc: starting...
W20260811 14:35:49.427713 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:35:49 dist-test-slave-twwt krb5kdc[12573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:49Z Disabled control of system clock
WARNING: no policy specified for oryx/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.126.keytab.
Entry for principal oryx/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.126.keytab.
I20260811 14:35:49.559690 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:34059
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.126.keytab
--principal=oryx/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:34059
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:49.643847 12592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:49.644106 12592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:49.644145 12592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:49.645274 12592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:49.645336 12592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:49.645360 12592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:49.645380 12592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:49.645402 12592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:49.646883 12592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:34059
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.126.keytab
--principal=oryx/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:34059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:49.647152 12592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:49.647375 12592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:49.652189 12598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:49.652208 12603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:49.652297 12599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:49.652506 12592 server_base.cc:1034] running on GCE node
I20260811 14:35:49.652746 12592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:49.652987 12592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:49.654160 12592 hybrid_clock.cc:648] HybridClock initialized: now 1786458949654140 us; error 33 us; skew 500 ppm
Aug 11 14:35:49 dist-test-slave-twwt krb5kdc[12573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458949, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:49.656929 12592 init.cc:377] Logged in from keytab as oryx/127.2.103.126@KRBTEST.COM (short username oryx)
I20260811 14:35:49.657471 12592 webserver.cc:460] Webserver started at http://127.2.103.126:34927/ using document root <none> and password file <none>
I20260811 14:35:49.657719 12592 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:49.657792 12592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:49.657963 12592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:49.659250 12592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "4d882bd3213f40e182237f47cbaefda5"
format_stamp: "Formatted at 2026-08-11 14:35:49 on dist-test-slave-twwt"
server_key: "c352ea7a300c76a2004223f8ed4bab1b"
server_key_iv: "ae83dd763b8b14afab5cbcd06de17e1f"
server_key_version: "encryptionkey@0"
I20260811 14:35:49.659672 12592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "4d882bd3213f40e182237f47cbaefda5"
format_stamp: "Formatted at 2026-08-11 14:35:49 on dist-test-slave-twwt"
server_key: "c352ea7a300c76a2004223f8ed4bab1b"
server_key_iv: "ae83dd763b8b14afab5cbcd06de17e1f"
server_key_version: "encryptionkey@0"
I20260811 14:35:49.661162 12592 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:35:49.661960 12615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:49.662199 12592 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:49.662329 12592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "4d882bd3213f40e182237f47cbaefda5"
format_stamp: "Formatted at 2026-08-11 14:35:49 on dist-test-slave-twwt"
server_key: "c352ea7a300c76a2004223f8ed4bab1b"
server_key_iv: "ae83dd763b8b14afab5cbcd06de17e1f"
server_key_version: "encryptionkey@0"
I20260811 14:35:49.662412 12592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:49.677129 12592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:49.679600 12592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:49.679782 12592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:49.684361 12592 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:34059
I20260811 14:35:49.684428 12699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:34059 every 8 connection(s)
I20260811 14:35:49.684945 12592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:49.685340 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 12592
I20260811 14:35:49.685461 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:35:49.685668 2461 external_mini_cluster.cc:1428] Setting key e978c0501a265c882a6809d2c7618131
I20260811 14:35:49.686170 12700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:49.689659 12700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5: Bootstrap starting.
I20260811 14:35:49.690354 12700 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:49.690732 12700 log.cc:824] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:49.691679 12700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5: No bootstrap required, opened a new log
Aug 11 14:35:49 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM
I20260811 14:35:49.694049 12700 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } }
I20260811 14:35:49.694247 12700 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:49.694303 12700 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d882bd3213f40e182237f47cbaefda5, State: Initialized, Role: FOLLOWER
I20260811 14:35:49.694438 12700 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } }
I20260811 14:35:49.694511 12700 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:49.694550 12700 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:49.694598 12700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:49.695178 12700 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } }
I20260811 14:35:49.695300 12700 leader_election.cc:302] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d882bd3213f40e182237f47cbaefda5; no voters:
I20260811 14:35:49.695497 12700 leader_election.cc:288] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:49.695531 12706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:49.695659 12706 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [term 1 LEADER]: Becoming Leader. State: Replica: 4d882bd3213f40e182237f47cbaefda5, State: Running, Role: LEADER
I20260811 14:35:49.695802 12700 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:49.695806 12706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } }
I20260811 14:35:49.696522 12707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d882bd3213f40e182237f47cbaefda5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } }
I20260811 14:35:49.696542 12708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d882bd3213f40e182237f47cbaefda5. Latest consensus state: current_term: 1 leader_uuid: "4d882bd3213f40e182237f47cbaefda5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } }
I20260811 14:35:49.696691 12707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:49.696962 12708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:49.697435 12722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:49.697325 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:49.686258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54962 (local address 127.2.103.126:34059)
0811 14:35:49.686454 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:49.686463 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:49.686497 (+ 34us) server_negotiation.cc:407] Connection header received
0811 14:35:49.687010 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:49.687018 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:49.687068 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:49.687186 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:49.688033 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:49.688939 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:49.689995 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:49.690411 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:49.690537 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:49.693258 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:49.693263 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:49.693263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:49.693277 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:49.695165 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:49.695673 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:49.695677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:49.695677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:49.695787 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:49.696037 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:49.696037 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:49.696038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:49.696323 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:49.696476 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:49.696943 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 14:35:49.697101 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":96,"threads_started":1}
I20260811 14:35:49.698207 12722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:49.699718 12722 catalog_manager.cc:1347] Generated new cluster ID: 2af9f7732d9744fb80bf2acaaec01923
I20260811 14:35:49.699772 12722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:35:49.723170 12722 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:49.723795 12722 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:49.733237 12722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5: Generated new TSK 0
I20260811 14:35:49.733457 12722 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.65.keytab.
Entry for principal oryx/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.65.keytab.
I20260811 14:35:49.768115 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:34059
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.65.keytab
--principal=oryx/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:49.875638 12746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:49.875839 12746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:49.875861 12746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:49.876982 12746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:49.877024 12746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:49.877051 12746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:35:49.878451 12746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.65.keytab
--principal=oryx/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:34059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:49.878695 12746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:49.878901 12746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:49.883957 12758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:49.884088 12746 server_base.cc:1034] running on GCE node
W20260811 14:35:49.883958 12754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:49.883994 12755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:49.884406 12746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:49.884634 12746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:49.885773 12746 hybrid_clock.cc:648] HybridClock initialized: now 1786458949885759 us; error 37 us; skew 500 ppm
Aug 11 14:35:49 dist-test-slave-twwt krb5kdc[12573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458949, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:49.888401 12746 init.cc:377] Logged in from keytab as oryx/127.2.103.65@KRBTEST.COM (short username oryx)
I20260811 14:35:49.888851 12746 webserver.cc:460] Webserver started at http://127.2.103.65:39897/ using document root <none> and password file <none>
I20260811 14:35:49.889106 12746 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:49.889150 12746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:49.889314 12746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:49.890429 12746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "d6025d4830f2498cb4f8430ac71bc2d0"
format_stamp: "Formatted at 2026-08-11 14:35:49 on dist-test-slave-twwt"
server_key: "a1236778cbe6abccb240a20fca8b6356"
server_key_iv: "4966cb62d98eb294b1e640e560e3f6c4"
server_key_version: "encryptionkey@0"
I20260811 14:35:49.890863 12746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "d6025d4830f2498cb4f8430ac71bc2d0"
format_stamp: "Formatted at 2026-08-11 14:35:49 on dist-test-slave-twwt"
server_key: "a1236778cbe6abccb240a20fca8b6356"
server_key_iv: "4966cb62d98eb294b1e640e560e3f6c4"
server_key_version: "encryptionkey@0"
I20260811 14:35:49.892378 12746 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:35:49.893182 12770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:49.893388 12746 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:35:49.893458 12746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "d6025d4830f2498cb4f8430ac71bc2d0"
format_stamp: "Formatted at 2026-08-11 14:35:49 on dist-test-slave-twwt"
server_key: "a1236778cbe6abccb240a20fca8b6356"
server_key_iv: "4966cb62d98eb294b1e640e560e3f6c4"
server_key_version: "encryptionkey@0"
I20260811 14:35:49.893538 12746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:49.923334 12746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:49.925649 12746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:49.925809 12746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:49.926091 12746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:49.926440 12746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:49.926473 12746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:49.926517 12746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:49.926544 12746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:49.932545 12746 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:45703
I20260811 14:35:49.932598 12973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:45703 every 8 connection(s)
I20260811 14:35:49.932938 12746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:35:49.935454 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 12746
I20260811 14:35:49.935585 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:35:49.935777 2461 external_mini_cluster.cc:1428] Setting key 8b094d52e1cc81e6986a8825e0a1497c
Aug 11 14:35:49 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458949, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM
I20260811 14:35:49.943665 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:49.933642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:44131 (local address 127.2.103.126:34059)
0811 14:35:49.933686 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:49.933689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:49.933908 (+ 219us) server_negotiation.cc:407] Connection header received
0811 14:35:49.934447 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:49.934448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:49.934460 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:49.934520 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:49.935519 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:49.936174 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:49.937256 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:49.937483 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:49.937567 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:49.940284 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:49.940286 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:49.940287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:49.940294 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:49.941916 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:49.942489 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:49.942490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:49.942491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:49.942547 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:49.942895 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:49.942896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:49.942896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:49.943035 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:49.943102 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:49.943514 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 14:35:49.943593 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:35:49.943982 12977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:49.933613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.65:44131)
0811 14:35:49.933785 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:49.933822 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:49.934334 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:49.934594 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:49.934598 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:49.935214 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:49.935416 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:49.935422 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:49.936273 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:49.936274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:49.937161 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:49.937164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:49.937580 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:49.937582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:49.937715 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:49.938294 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:49.938307 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:49.940148 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:49.942011 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:49.942021 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:49.942024 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:49.942407 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:49.942635 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:49.942636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:49.942637 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:49.942803 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:49.943111 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:49.943119 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:49.943315 (+ 196us) client_negotiation.cc:769] Sending connection context
0811 14:35:49.943443 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 14:35:49.943637 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":78,"threads_started":1}
I20260811 14:35:49.944590 12974 heartbeater.cc:347] Connected to a master server at 127.2.103.126:34059
I20260811 14:35:49.944696 12974 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:49.944913 12974 heartbeater.cc:511] Master 127.2.103.126:34059 requested a full tablet report, sending...
I20260811 14:35:49.945477 12643 ts_manager.cc:194] Registered new tserver with Master: d6025d4830f2498cb4f8430ac71bc2d0 (127.2.103.65:45703)
WARNING: no policy specified for oryx/127.2.103.66@KRBTEST.COM; defaulting to no policy
I20260811 14:35:49.946110 12643 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:44131
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.103.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.66.keytab.
Entry for principal oryx/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.66.keytab.
I20260811 14:35:49.973860 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:34059
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.66.keytab
--principal=oryx/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:50.058797 12985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:50.059008 12985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:50.059033 12985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:50.060240 12985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:50.060289 12985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:50.060320 12985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:35:50.061708 12985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.66.keytab
--principal=oryx/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:34059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:50.061921 12985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:50.062104 12985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:50.067207 12999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:50.067215 12997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:50.067374 12985 server_base.cc:1034] running on GCE node
W20260811 14:35:50.067257 12996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:50.067665 12985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:50.067920 12985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:50.069079 12985 hybrid_clock.cc:648] HybridClock initialized: now 1786458950069070 us; error 39 us; skew 500 ppm
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458950, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:50.071635 12985 init.cc:377] Logged in from keytab as oryx/127.2.103.66@KRBTEST.COM (short username oryx)
I20260811 14:35:50.072114 12985 webserver.cc:460] Webserver started at http://127.2.103.66:46185/ using document root <none> and password file <none>
I20260811 14:35:50.072343 12985 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:50.072408 12985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:50.072566 12985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:50.073650 12985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "3d80c3098fb34a9b97eda8e7427eb3d6"
format_stamp: "Formatted at 2026-08-11 14:35:50 on dist-test-slave-twwt"
server_key: "5d392d48d280500d670b35a4f7e62506"
server_key_iv: "cbd9c2db7b8d6ed302b1e30e76bb37ed"
server_key_version: "encryptionkey@0"
I20260811 14:35:50.074031 12985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "3d80c3098fb34a9b97eda8e7427eb3d6"
format_stamp: "Formatted at 2026-08-11 14:35:50 on dist-test-slave-twwt"
server_key: "5d392d48d280500d670b35a4f7e62506"
server_key_iv: "cbd9c2db7b8d6ed302b1e30e76bb37ed"
server_key_version: "encryptionkey@0"
I20260811 14:35:50.075534 12985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:35:50.076292 13013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:50.076457 12985 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:50.076557 12985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "3d80c3098fb34a9b97eda8e7427eb3d6"
format_stamp: "Formatted at 2026-08-11 14:35:50 on dist-test-slave-twwt"
server_key: "5d392d48d280500d670b35a4f7e62506"
server_key_iv: "cbd9c2db7b8d6ed302b1e30e76bb37ed"
server_key_version: "encryptionkey@0"
I20260811 14:35:50.076637 12985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:50.095065 12985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:50.097525 12985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:50.097692 12985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:50.097957 12985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:50.098282 12985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:50.098341 12985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:50.098387 12985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:50.098426 12985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:50.104451 12985 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:44753
I20260811 14:35:50.104514 13188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:44753 every 8 connection(s)
I20260811 14:35:50.104887 12985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:35:50.110224 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 12985
I20260811 14:35:50.110352 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:35:50.110590 2461 external_mini_cluster.cc:1428] Setting key 77130762f8aa7a274d211f8eddcc0f2c
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458950, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM
I20260811 14:35:50.115430 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.105731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:55817 (local address 127.2.103.126:34059)
0811 14:35:50.105801 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.105804 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.106053 (+ 249us) server_negotiation.cc:407] Connection header received
0811 14:35:50.106616 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.106618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.106632 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.106694 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.107711 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.108216 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.109246 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.109455 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.109532 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.112276 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.112277 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.112277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.112285 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.113967 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.114365 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.114367 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.114367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.114436 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.114744 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.114745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.114745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.114884 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.114941 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.115285 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.115364 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 14:35:50.115733 13195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.105709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.66:55817)
0811 14:35:50.105923 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:50.105961 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:50.106492 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:50.106783 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:50.106787 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:50.107388 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:50.107591 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.107596 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.108314 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.108316 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.109122 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.109124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.109557 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.109558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.109686 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.110354 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:50.110365 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:50.112183 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:50.114056 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.114063 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.114065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.114292 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.114515 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.114516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.114516 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.114686 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.114954 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:50.114961 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:50.115138 (+ 177us) client_negotiation.cc:769] Sending connection context
0811 14:35:50.115255 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 14:35:50.115446 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":128,"threads_started":1}
I20260811 14:35:50.116302 13189 heartbeater.cc:347] Connected to a master server at 127.2.103.126:34059
I20260811 14:35:50.116421 13189 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:50.116640 13189 heartbeater.cc:511] Master 127.2.103.126:34059 requested a full tablet report, sending...
I20260811 14:35:50.117085 12643 ts_manager.cc:194] Registered new tserver with Master: 3d80c3098fb34a9b97eda8e7427eb3d6 (127.2.103.66:44753)
I20260811 14:35:50.117461 12643 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:55817
WARNING: no policy specified for oryx/127.2.103.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.103.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.67.keytab.
Entry for principal oryx/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.67.keytab.
I20260811 14:35:50.137925 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:34059
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.67.keytab
--principal=oryx/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:50.227326 13199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:50.227572 13199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:50.227612 13199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:50.228768 13199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:50.228833 13199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:50.228876 13199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:35:50.230384 13199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.67.keytab
--principal=oryx/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:34059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:50.230684 13199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:50.230923 13199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:50.236235 13208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:50.236328 13214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:50.236421 13199 server_base.cc:1034] running on GCE node
W20260811 14:35:50.236326 13209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:50.236720 13199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:50.236982 13199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:50.238142 13199 hybrid_clock.cc:648] HybridClock initialized: now 1786458950238129 us; error 40 us; skew 500 ppm
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458950, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:50.240856 13199 init.cc:377] Logged in from keytab as oryx/127.2.103.67@KRBTEST.COM (short username oryx)
I20260811 14:35:50.241320 13199 webserver.cc:460] Webserver started at http://127.2.103.67:38629/ using document root <none> and password file <none>
I20260811 14:35:50.241565 13199 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:50.241612 13199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:50.241776 13199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:50.242966 13199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "570caddfa3a84f0682765c59723893dd"
format_stamp: "Formatted at 2026-08-11 14:35:50 on dist-test-slave-twwt"
server_key: "98e919a0399ec0c4dc8ee5994a5af7f9"
server_key_iv: "ceb2631708af3a166e0d7f09653fe517"
server_key_version: "encryptionkey@0"
I20260811 14:35:50.243335 13199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "570caddfa3a84f0682765c59723893dd"
format_stamp: "Formatted at 2026-08-11 14:35:50 on dist-test-slave-twwt"
server_key: "98e919a0399ec0c4dc8ee5994a5af7f9"
server_key_iv: "ceb2631708af3a166e0d7f09653fe517"
server_key_version: "encryptionkey@0"
I20260811 14:35:50.244910 13199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:35:50.245710 13224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:50.246021 13199 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:50.246116 13199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "570caddfa3a84f0682765c59723893dd"
format_stamp: "Formatted at 2026-08-11 14:35:50 on dist-test-slave-twwt"
server_key: "98e919a0399ec0c4dc8ee5994a5af7f9"
server_key_iv: "ceb2631708af3a166e0d7f09653fe517"
server_key_version: "encryptionkey@0"
I20260811 14:35:50.246178 13199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:50.262087 13199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:50.264397 13199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:50.264560 13199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:50.264786 13199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:50.265123 13199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:50.265175 13199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:50.265229 13199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:50.265250 13199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:50.271426 13199 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:41451
I20260811 14:35:50.271513 13397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:41451 every 8 connection(s)
I20260811 14:35:50.271890 13199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:35:50.274072 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 13199
I20260811 14:35:50.274176 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:35:50.274358 2461 external_mini_cluster.cc:1428] Setting key b2c3338a13b4eaeef6a4cfb36070ddd3
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458950, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM
I20260811 14:35:50.281591 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.272657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:41263 (local address 127.2.103.126:34059)
0811 14:35:50.272711 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.272713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.272917 (+ 204us) server_negotiation.cc:407] Connection header received
0811 14:35:50.273402 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.273404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.273414 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.273467 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.274695 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.275213 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.276198 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.276414 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.276498 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.278722 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.278724 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.278724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.278731 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.280198 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.280702 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.280703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.280703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.280755 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.281017 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.281018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.281018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.281125 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.281178 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.281457 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.281525 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:35:50.281796 13407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.272632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.67:41263)
0811 14:35:50.272809 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:50.272844 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:50.273316 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:50.273523 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:50.273526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:50.274273 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:50.274572 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.274579 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.275312 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.275313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.276108 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.276110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.276510 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.276511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.276635 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.277157 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:50.277166 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:50.278642 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:50.280270 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.280291 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.280293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.280627 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.280837 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.280838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.280838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.280953 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.281196 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:50.281202 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:50.281333 (+ 131us) client_negotiation.cc:769] Sending connection context
0811 14:35:50.281426 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 14:35:50.281550 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":111,"threads_started":1}
I20260811 14:35:50.282361 13403 heartbeater.cc:347] Connected to a master server at 127.2.103.126:34059
I20260811 14:35:50.282488 13403 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:50.282693 13403 heartbeater.cc:511] Master 127.2.103.126:34059 requested a full tablet report, sending...
I20260811 14:35:50.283085 12643 ts_manager.cc:194] Registered new tserver with Master: 570caddfa3a84f0682765c59723893dd (127.2.103.67:41451)
I20260811 14:35:50.283471 12643 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:41263
I20260811 14:35:50.285971 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.2.103.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.103.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.103.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.125.keytab.
Entry for principal oryx/127.2.103.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.125.keytab.
I20260811 14:35:50.314078 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.125:40859
--webserver_interface=127.2.103.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.125.keytab
--principal=oryx/127.2.103.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:34059,127.2.103.125:40859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:50.402449 13411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:50.402735 13411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:50.402779 13411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:50.404027 13411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:50.404100 13411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:50.404124 13411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:50.404143 13411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:50.404170 13411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:50.405768 13411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:34059,127.2.103.125:40859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.125.keytab
--principal=oryx/127.2.103.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.125:40859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.103.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
W20260811 14:35:50.405936 13411 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.2.103.126:34059,127.2.103.125:40859'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 14:35:50.406105 13411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:50.406347 13411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:50.411608 13422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:50.411659 13420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:50.411674 13411 server_base.cc:1034] running on GCE node
W20260811 14:35:50.411617 13419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:50.412117 13411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:50.412391 13411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:50.413534 13411 hybrid_clock.cc:648] HybridClock initialized: now 1786458950413523 us; error 30 us; skew 500 ppm
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458950, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:50.416208 13411 init.cc:377] Logged in from keytab as oryx/127.2.103.125@KRBTEST.COM (short username oryx)
I20260811 14:35:50.416689 13411 webserver.cc:460] Webserver started at http://127.2.103.125:43449/ using document root <none> and password file <none>
I20260811 14:35:50.416887 13411 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:50.416927 13411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:50.417018 13411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:50.418151 13411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/data/instance:
uuid: "2abc19ebb22040029f5122671c37a3d9"
format_stamp: "Formatted at 2026-08-11 14:35:50 on dist-test-slave-twwt"
server_key: "9e0cf95de2bb2c8be3dc871c0e16badf"
server_key_iv: "ee7256f6fd2c3215890c64f8be3f569a"
server_key_version: "encryptionkey@0"
I20260811 14:35:50.418473 13411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/wal/instance:
uuid: "2abc19ebb22040029f5122671c37a3d9"
format_stamp: "Formatted at 2026-08-11 14:35:50 on dist-test-slave-twwt"
server_key: "9e0cf95de2bb2c8be3dc871c0e16badf"
server_key_iv: "ee7256f6fd2c3215890c64f8be3f569a"
server_key_version: "encryptionkey@0"
I20260811 14:35:50.419905 13411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:35:50.420717 13441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:50.420936 13411 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:35:50.421029 13411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/wal
uuid: "2abc19ebb22040029f5122671c37a3d9"
format_stamp: "Formatted at 2026-08-11 14:35:50 on dist-test-slave-twwt"
server_key: "9e0cf95de2bb2c8be3dc871c0e16badf"
server_key_iv: "ee7256f6fd2c3215890c64f8be3f569a"
server_key_version: "encryptionkey@0"
I20260811 14:35:50.421104 13411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:50.434921 13411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:50.437260 13411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:50.437427 13411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:50.442215 13411 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.125:40859
I20260811 14:35:50.442282 13521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.125:40859 every 8 connection(s)
I20260811 14:35:50.442664 13411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/data/info.pb
I20260811 14:35:50.443765 13522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:50.445411 13522 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:35:50.450783 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 13411
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458950, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.125@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM
I20260811 14:35:50.455360 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.446148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54972 (local address 127.2.103.126:34059)
0811 14:35:50.446236 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.446239 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.446431 (+ 192us) server_negotiation.cc:407] Connection header received
0811 14:35:50.447003 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.447004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.447015 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.447078 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.447702 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.448258 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.449397 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.449592 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.449661 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.452075 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.452077 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.452077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.452086 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.453772 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.454279 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.454280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.454280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.454337 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.454641 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.454642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.454643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.454795 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.454906 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.455153 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.455239 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 14:35:50.455470 13524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.446112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.0.0.1:54972)
0811 14:35:50.446309 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:50.446346 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:50.446879 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:50.447152 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:50.447156 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:50.447417 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:50.447614 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.447618 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.448339 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.448339 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.449283 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.449287 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.449678 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.449679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.449795 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.450340 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:50.450349 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:50.451981 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:50.453863 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.453869 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.453871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.454194 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.454401 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.454401 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.454402 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.454542 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.454900 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:50.454906 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:50.455025 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 14:35:50.455114 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 14:35:50.455241 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":111,"threads_started":1}
I20260811 14:35:50.456193 13522 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.103.125@KRBTEST.COM
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458950, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.125@KRBTEST.COM for oryx/127.2.103.125@KRBTEST.COM
I20260811 14:35:50.461408 13527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.451429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42006 (local address 127.2.103.125:40859)
0811 14:35:50.451633 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.451638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.451723 (+ 85us) server_negotiation.cc:407] Connection header received
0811 14:35:50.451753 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.451755 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.451780 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.451901 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.452555 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.453829 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.454882 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.455274 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.455353 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.458315 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.458324 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.458325 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.458332 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.460081 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.460548 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.460551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.460552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.460617 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.460806 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.460808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.460808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.460977 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.461152 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.461211 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.461284 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":104,"threads_started":1}
I20260811 14:35:50.464005 13524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.456485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:40859 (local address 127.0.0.1:42022)
0811 14:35:50.456525 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:50.456534 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:50.456571 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:50.456760 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:50.456762 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:50.456909 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:50.456968 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.456970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.457978 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.457979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.458490 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.458492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.459109 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.459110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.459180 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.459540 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:50.459544 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:50.460813 (+ 1269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:50.463183 (+ 2370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.463183 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.463184 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.463359 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.463536 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.463537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.463537 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.463587 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.463783 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:50.463784 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:50.463817 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 14:35:50.463869 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 14:35:50.463943 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260811 14:35:50.464157 13528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.456519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42022 (local address 127.2.103.125:40859)
0811 14:35:50.456660 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.456661 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.456666 (+ 5us) server_negotiation.cc:407] Connection header received
0811 14:35:50.456679 (+ 13us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.456679 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.456685 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.456731 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.457072 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.457848 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.458808 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.459017 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.459107 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.460901 (+ 1794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.460903 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.460903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.460909 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.463101 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.463425 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.463425 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.463426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.463480 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.463626 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.463627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.463627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.463711 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.463798 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.463895 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.463991 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":127,"thread_start_us":103,"threads_started":1}
I20260811 14:35:50.465514 13522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9: Bootstrap starting.
I20260811 14:35:50.466111 13522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:50.466403 13522 log.cc:824] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:50.467265 13522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9: No bootstrap required, opened a new log
I20260811 14:35:50.468649 13522 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } }
I20260811 14:35:50.468777 13522 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:50.468801 13522 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2abc19ebb22040029f5122671c37a3d9, State: Initialized, Role: FOLLOWER
I20260811 14:35:50.468932 13522 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } }
I20260811 14:35:50.469208 13531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } } }
I20260811 14:35:50.469288 13531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:35:50.469323 13522 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 14:35:50.471139 13545 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:35:50.471213 13545 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:35:50.472851 13411 master_runner.cc:428] Detected that this master 2abc19ebb22040029f5122671c37a3d9 is joining an existing cluster
I20260811 14:35:50.472929 13411 master_runner.cc:432] Initiating AddMaster RPC to add 127.2.103.125:40859
I20260811 14:35:50.481513 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.473941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54988 (local address 127.2.103.126:34059)
0811 14:35:50.474013 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.474016 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.474310 (+ 294us) server_negotiation.cc:407] Connection header received
0811 14:35:50.474386 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.474387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.474400 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.474452 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.475034 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.475570 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.476647 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.476894 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.476927 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.478230 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.478232 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.478232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.478239 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.479992 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.480544 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.480545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.480546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.480606 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.480900 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.480901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.480901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.481043 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.481088 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.481318 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.481407 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 14:35:50.481750 13528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.474114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42032 (local address 127.2.103.125:40859)
0811 14:35:50.474196 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.474198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.474351 (+ 153us) server_negotiation.cc:407] Connection header received
0811 14:35:50.474421 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.474422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.474430 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.474477 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.475010 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.475756 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.476729 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.477056 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.477131 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.478209 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.478211 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.478212 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.478219 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.480490 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.480877 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.480879 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.480879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.480935 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.481159 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.481160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.481160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.481295 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.481360 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.481527 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.481653 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":62}
I20260811 14:35:50.481962 13562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.473917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:40859 (local address 127.0.0.1:42032)
0811 14:35:50.474288 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:50.474300 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:50.474367 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:50.474494 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:50.474495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:50.474823 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:50.474929 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.474931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.475850 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.475851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.476650 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.476652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.477141 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.477142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.477208 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.477617 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:50.477622 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:50.478120 (+ 498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:50.480568 (+ 2448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.480569 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.480570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.480795 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.481006 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.481007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.481007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.481088 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.481363 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:50.481364 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:50.481416 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 14:35:50.481492 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 14:35:50.481863 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":3,"thread_start_us":214,"threads_started":1}
I20260811 14:35:50.482172 13561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.473900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.0.0.1:54988)
0811 14:35:50.474244 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:50.474255 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:50.474317 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:50.474499 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:50.474500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:50.474827 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:50.474929 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.474931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.475649 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.475650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.476546 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.476548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.477004 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.477005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.477048 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.477738 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:50.477743 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:50.478126 (+ 383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:50.480070 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.480071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.480072 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.480446 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.480692 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.480693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.480693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.480767 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.481144 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:50.481145 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:50.481192 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 14:35:50.481932 (+ 740us) client_negotiation.cc:240] Negotiation successful
0811 14:35:50.482055 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":20,"thread_start_us":341,"threads_started":1}
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458949, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.126@KRBTEST.COM for oryx/127.2.103.125@KRBTEST.COM
I20260811 14:35:50.492286 13528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.484482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42038 (local address 127.2.103.125:40859)
0811 14:35:50.484524 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.484526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.484669 (+ 143us) server_negotiation.cc:407] Connection header received
0811 14:35:50.484761 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.484762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.484770 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.484821 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.485317 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.485742 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.486892 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.487083 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.487145 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.489491 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.489493 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.489493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.489500 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.491018 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.491529 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.491531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.491531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.491590 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.491805 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.491806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.491806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.491909 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.491962 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.492158 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.492223 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:35:50.492388 13563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.484388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:40859 (local address 127.0.0.1:42038)
0811 14:35:50.484548 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:50.484602 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:50.484684 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:50.484826 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:50.484829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:50.485174 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:50.485238 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.485240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.485816 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.485817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.486806 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.486810 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.487162 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.487163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.487250 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.487749 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:50.487755 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:50.489371 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:50.491143 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.491151 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.491154 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.491441 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.491662 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.491663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.491664 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.491737 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.491971 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:50.491978 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:50.492038 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 14:35:50.492132 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 14:35:50.492258 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260811 14:35:50.493002 12643 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.2.103.125:40859: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2abc19ebb22040029f5122671c37a3d9"
member_type: NON_VOTER
last_known_addr {
host: "127.2.103.125"
port: 40859
}
attrs {
promote: true
}
}
dest_uuid: "4d882bd3213f40e182237f47cbaefda5"
cas_config_opid_index: -1
I20260811 14:35:50.493408 12643 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } }
I20260811 14:35:50.493957 13565 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2abc19ebb22040029f5122671c37a3d9 (127.2.103.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } } }
I20260811 14:35:50.494308 13481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:50.495129 13565 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.2.103.125:40859
I20260811 14:35:50.495222 13481 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 1 FOLLOWER]: Refusing update from remote peer 4d882bd3213f40e182237f47cbaefda5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 14:35:50.495216 13565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4d882bd3213f40e182237f47cbaefda5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } } }
I20260811 14:35:50.495316 13565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:50.495589 13565 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:50.495633 13411 master.cc:537] Master@127.2.103.125:40859 shutting down...
W20260811 14:35:50.496416 12620 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 14:35:50.496855 12620 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 -> Peer 2abc19ebb22040029f5122671c37a3d9 (127.2.103.125:40859): Couldn't send request to peer 2abc19ebb22040029f5122671c37a3d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 14:35:50.498749 13411 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:50.498860 13411 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:50.498915 13411 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9: stopping tablet replica
I20260811 14:35:50.501966 13411 master.cc:552] Master@127.2.103.125:40859 shutdown complete.
I20260811 14:35:50.502017 13411 master_runner.cc:305] Clearing existing system tablet
I20260811 14:35:50.502154 13411 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:50.502204 13411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:50.503001 13411 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 14:35:50.503522 13574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:50.503654 13411 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:50.503717 13411 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:50.503742 13411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/wal
uuid: "2abc19ebb22040029f5122671c37a3d9"
format_stamp: "Formatted at 2026-08-11 14:35:50 on dist-test-slave-twwt"
server_key: "9e0cf95de2bb2c8be3dc871c0e16badf"
server_key_iv: "ee7256f6fd2c3215890c64f8be3f569a"
server_key_version: "encryptionkey@0"
I20260811 14:35:50.504628 13411 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:50.506112 13411 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 14:35:50.506188 13411 log.cc:1196] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 14:35:50.506410 13411 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9: Deleting consensus metadata
I20260811 14:35:50.506693 13411 master_runner.cc:315] Copying system tablet from 127.2.103.126:34059
I20260811 14:35:50.507366 13411 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9: tablet copy: Beginning tablet copy session from remote peer at address 127.2.103.126:34059
I20260811 14:35:50.513865 13585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.507743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.0.0.1:54998)
0811 14:35:50.507877 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:50.507887 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:50.507942 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:50.508084 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:50.508085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:50.508241 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:50.508299 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.508300 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.509253 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.509254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.509821 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.509823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.510302 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.510303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.510351 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.510874 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:50.510878 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:50.511186 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:50.512928 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.512929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.512929 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.513103 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.513304 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.513304 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.513305 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.513357 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.513624 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:50.513624 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:50.513661 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 14:35:50.513709 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 14:35:50.513791 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":71,"threads_started":1}
I20260811 14:35:50.513921 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.507803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54998 (local address 127.2.103.126:34059)
0811 14:35:50.507860 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.507863 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.507932 (+ 69us) server_negotiation.cc:407] Connection header received
0811 14:35:50.507991 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.507992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.508003 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.508082 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.508363 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.509153 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.509918 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.510229 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.510303 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.511279 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.511281 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.511282 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.511290 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.512851 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.513173 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.513174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.513175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.513239 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.513418 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.513419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.513420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.513557 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.513621 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.513742 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.513823 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 14:35:50.514578 12675 tablet_copy_service.cc:140] P 4d882bd3213f40e182237f47cbaefda5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2abc19ebb22040029f5122671c37a3d9 ({username='oryx', principal='oryx/127.2.103.125@KRBTEST.COM'} at 127.0.0.1:54998)
I20260811 14:35:50.514779 12675 tablet_copy_service.cc:161] P 4d882bd3213f40e182237f47cbaefda5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2abc19ebb22040029f5122671c37a3d9 at {username='oryx', principal='oryx/127.2.103.125@KRBTEST.COM'} at 127.0.0.1:54998: session id = 2abc19ebb22040029f5122671c37a3d9-00000000000000000000000000000000
I20260811 14:35:50.515517 12675 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 14:35:50.516181 13411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 14:35:50.517614 13411 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9: tablet copy: Starting download of 0 data blocks...
I20260811 14:35:50.517774 13411 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9: tablet copy: Starting download of 1 WAL segments...
I20260811 14:35:50.518771 13411 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 14:35:50.519837 12675 tablet_copy_service.cc:342] P 4d882bd3213f40e182237f47cbaefda5: Request end of tablet copy session 2abc19ebb22040029f5122671c37a3d9-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.2.103.125@KRBTEST.COM'} at 127.0.0.1:54998
I20260811 14:35:50.519922 12675 tablet_copy_service.cc:433] P 4d882bd3213f40e182237f47cbaefda5: ending tablet copy session 2abc19ebb22040029f5122671c37a3d9-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2abc19ebb22040029f5122671c37a3d9
I20260811 14:35:50.522521 13411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:50.523833 13586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:50.523871 13587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:50.527776 13592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:50.527755 13411 server_base.cc:1034] running on GCE node
I20260811 14:35:50.527943 13411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:50.528115 13411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:50.529235 13411 hybrid_clock.cc:648] HybridClock initialized: now 1786458950529223 us; error 30 us; skew 500 ppm
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458950, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:50.530659 13411 init.cc:377] Logged in from keytab as oryx/127.2.103.125@KRBTEST.COM (short username oryx)
I20260811 14:35:50.530997 13411 webserver.cc:460] Webserver started at http://127.2.103.125:44087/ using document root <none> and password file <none>
I20260811 14:35:50.531139 13411 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:50.531179 13411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:50.531848 13411 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:50.532550 13607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:50.532673 13411 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:50.532730 13411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/wal
uuid: "2abc19ebb22040029f5122671c37a3d9"
format_stamp: "Formatted at 2026-08-11 14:35:50 on dist-test-slave-twwt"
server_key: "9e0cf95de2bb2c8be3dc871c0e16badf"
server_key_iv: "ee7256f6fd2c3215890c64f8be3f569a"
server_key_version: "encryptionkey@0"
I20260811 14:35:50.532785 13411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:50.542212 13411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:50.543910 13411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:50.547250 13411 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.125:40859
I20260811 14:35:50.547645 13696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.125:40859 every 8 connection(s)
I20260811 14:35:50.547731 13411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-1/data/info.pb
I20260811 14:35:50.548629 13697 sys_catalog.cc:263] Verifying existing consensus state
I20260811 14:35:50.549054 13697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9: Bootstrap starting.
I20260811 14:35:50.551553 13697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:35:50.551782 13697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9: Bootstrap complete.
I20260811 14:35:50.551980 13697 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } }
I20260811 14:35:50.552067 13697 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2abc19ebb22040029f5122671c37a3d9, State: Initialized, Role: LEARNER
I20260811 14:35:50.552119 13697 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } }
I20260811 14:35:50.552322 13697 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:50.552296 13702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } } }
I20260811 14:35:50.552373 13702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [sys.catalog]: This master's current role is: LEARNER
I20260811 14:35:50.553783 13718 catalog_manager.cc:1259] Loaded cluster ID: 2af9f7732d9744fb80bf2acaaec01923
I20260811 14:35:50.553858 13718 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9: loading cluster ID for follower catalog manager: success
I20260811 14:35:50.554951 13718 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9: acquiring CA information for follower catalog manager: success
I20260811 14:35:50.555238 13718 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 14:35:50.570575 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.563052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42050 (local address 127.2.103.125:40859)
0811 14:35:50.563173 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.563175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.563182 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:35:50.563213 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.563213 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.563223 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.563275 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.564082 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.564823 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.565922 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.566122 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.566174 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.567731 (+ 1557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.567733 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.567733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.567740 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.569245 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.569797 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.569797 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.569798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.569850 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.570090 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.570091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.570091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.570192 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.570269 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.570423 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.570492 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for oryx/127.2.103.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.103.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.103.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.124.keytab.
Entry for principal oryx/127.2.103.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.124.keytab.
I20260811 14:35:50.599906 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.124:33113
--webserver_interface=127.2.103.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.124.keytab
--principal=oryx/127.2.103.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:34059,127.2.103.125:40859,127.2.103.124:33113
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:50.691761 13729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:50.692024 13729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:50.692065 13729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:50.693328 13729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:50.693400 13729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:50.693424 13729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:50.693455 13729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:50.693488 13729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:50.695184 13729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:34059,127.2.103.125:40859,127.2.103.124:33113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.124.keytab
--principal=oryx/127.2.103.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.124:33113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.103.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:50.695468 13729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:50.695708 13729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:50.701216 13736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:50.701148 13737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:50.701133 13742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:50.701419 13729 server_base.cc:1034] running on GCE node
I20260811 14:35:50.701650 13729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:50.701949 13729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:50.703110 13729 hybrid_clock.cc:648] HybridClock initialized: now 1786458950703090 us; error 46 us; skew 500 ppm
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458950, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:50.705675 13729 init.cc:377] Logged in from keytab as oryx/127.2.103.124@KRBTEST.COM (short username oryx)
I20260811 14:35:50.706185 13729 webserver.cc:460] Webserver started at http://127.2.103.124:40095/ using document root <none> and password file <none>
I20260811 14:35:50.706425 13729 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:50.706492 13729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:50.706687 13729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:50.707864 13729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/data/instance:
uuid: "c7e1ff4603b74cfa81ef09f40f4dc030"
format_stamp: "Formatted at 2026-08-11 14:35:50 on dist-test-slave-twwt"
server_key: "4ebe94b8fc91e1dd39488fc5bbcbe805"
server_key_iv: "7217b1dfdb39855ae54a7b1447906452"
server_key_version: "encryptionkey@0"
I20260811 14:35:50.708256 13729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/wal/instance:
uuid: "c7e1ff4603b74cfa81ef09f40f4dc030"
format_stamp: "Formatted at 2026-08-11 14:35:50 on dist-test-slave-twwt"
server_key: "4ebe94b8fc91e1dd39488fc5bbcbe805"
server_key_iv: "7217b1dfdb39855ae54a7b1447906452"
server_key_version: "encryptionkey@0"
I20260811 14:35:50.709764 13729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:50.710671 13754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:50.710896 13729 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:50.710999 13729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/wal
uuid: "c7e1ff4603b74cfa81ef09f40f4dc030"
format_stamp: "Formatted at 2026-08-11 14:35:50 on dist-test-slave-twwt"
server_key: "4ebe94b8fc91e1dd39488fc5bbcbe805"
server_key_iv: "7217b1dfdb39855ae54a7b1447906452"
server_key_version: "encryptionkey@0"
I20260811 14:35:50.711078 13729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:50.719179 13729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:50.721174 13729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:50.721359 13729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:50.728678 13729 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.124:33113
I20260811 14:35:50.728711 13841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.124:33113 every 8 connection(s)
I20260811 14:35:50.729099 13729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/data/info.pb
I20260811 14:35:50.730252 13842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:50.732091 13842 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:35:50.736500 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 13729
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458950, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.124@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM
I20260811 14:35:50.741825 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.732954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55012 (local address 127.2.103.126:34059)
0811 14:35:50.733035 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.733038 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.733157 (+ 119us) server_negotiation.cc:407] Connection header received
0811 14:35:50.733729 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.733731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.733741 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.733805 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.734488 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.735043 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.736031 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.736224 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.736282 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.738735 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.738737 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.738738 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.738746 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.740283 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.740741 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.740743 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.740743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.740809 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.741105 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.741107 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.741107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.741257 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.741339 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.741598 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.741693 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 14:35:50.741830 13845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.732847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.0.0.1:55012)
0811 14:35:50.733037 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:50.733073 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:50.733638 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:50.733886 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:50.733890 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:50.734145 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:50.734414 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.734419 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.735128 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.735129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.735955 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.735958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.736332 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.736333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.736453 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.737028 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:50.737041 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:50.738640 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:50.740355 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.740360 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.740362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.740667 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.740890 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.740891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.740891 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.741038 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.741331 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:50.741336 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:50.741458 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 14:35:50.741542 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 14:35:50.741661 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":90,"threads_started":1}
I20260811 14:35:50.742475 13842 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.103.124@KRBTEST.COM
I20260811 14:35:50.746649 13846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.737138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57846 (local address 127.2.103.124:33113)
0811 14:35:50.737310 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.737314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.737344 (+ 30us) server_negotiation.cc:407] Connection header received
0811 14:35:50.737366 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.737369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.737389 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.737474 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.738044 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.738963 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.740091 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.740463 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.740537 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.743340 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.743347 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.743347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.743353 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.745103 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.745548 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.745552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.745552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.745764 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.746015 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.746016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.746017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.746208 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.746327 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.746465 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.746530 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":92,"threads_started":1}
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458950, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.124@KRBTEST.COM for oryx/127.2.103.125@KRBTEST.COM
I20260811 14:35:50.749715 13845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.742758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:40859 (local address 127.0.0.1:42062)
0811 14:35:50.742805 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:50.742817 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:50.742885 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:50.743033 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:50.743034 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:50.743145 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:50.743202 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.743203 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.743866 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.743867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.744654 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.744657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.745037 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.745038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.745124 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.745535 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:50.745540 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:50.747196 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:50.748780 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.748781 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.748781 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.748964 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.749165 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.749167 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.749167 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.749232 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.749473 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:50.749474 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:50.749514 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 14:35:50.749568 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 14:35:50.749648 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 14:35:50.749737 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.742847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42062 (local address 127.2.103.125:40859)
0811 14:35:50.742907 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.742910 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.742919 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:35:50.742949 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.742949 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.742964 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.743032 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.743281 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.743781 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.744748 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.744966 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.745018 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.747283 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.747285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.747285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.747292 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.748649 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.749030 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.749031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.749031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.749086 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.749298 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.749298 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.749298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.749405 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.749458 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.749592 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.749673 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 14:35:50.750219 13842 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458950, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.124@KRBTEST.COM for oryx/127.2.103.124@KRBTEST.COM
I20260811 14:35:50.756592 13845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.750515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33113 (local address 127.0.0.1:57850)
0811 14:35:50.750562 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:50.750572 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:50.750632 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:50.750746 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:50.750747 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:50.750888 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:50.750918 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.750919 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.751551 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.751552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.752048 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.752051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.752437 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.752438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.752485 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.752902 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:50.752906 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:50.754168 (+ 1262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:50.755556 (+ 1388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.755557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.755557 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.755723 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.755924 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.755925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.755925 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.755980 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.756237 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:50.756238 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:50.756275 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 14:35:50.756347 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 14:35:50.756453 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 14:35:50.756597 13846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.750577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57850 (local address 127.2.103.124:33113)
0811 14:35:50.750649 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.750652 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.750659 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:35:50.750679 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.750680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.750689 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.750744 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.750988 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.751474 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.752115 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.752368 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.752441 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.754254 (+ 1813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.754256 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.754256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.754263 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.755463 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.755803 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.755804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.755804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.755861 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.756048 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.756048 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.756049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.756163 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.756234 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.756390 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.756504 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":22}
I20260811 14:35:50.758327 13842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: Bootstrap starting.
I20260811 14:35:50.759354 13842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:50.759644 13842 log.cc:824] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:50.760463 13842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: No bootstrap required, opened a new log
I20260811 14:35:50.762240 13842 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } }
I20260811 14:35:50.762382 13842 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:50.762406 13842 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7e1ff4603b74cfa81ef09f40f4dc030, State: Initialized, Role: FOLLOWER
I20260811 14:35:50.762557 13842 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } }
I20260811 14:35:50.762972 13848 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } } }
I20260811 14:35:50.763057 13842 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:50.763084 13848 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 14:35:50.765194 13863 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:35:50.765269 13863 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:35:50.768716 13729 master_runner.cc:428] Detected that this master c7e1ff4603b74cfa81ef09f40f4dc030 is joining an existing cluster
I20260811 14:35:50.768824 13729 master_runner.cc:432] Initiating AddMaster RPC to add 127.2.103.124:33113
I20260811 14:35:50.778564 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.770718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42066 (local address 127.2.103.125:40859)
0811 14:35:50.770775 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.770777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.770786 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:35:50.770811 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.770812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.770824 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.770909 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.771476 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.772063 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.773374 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.773671 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.773710 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.775397 (+ 1687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.775399 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.775399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.775409 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.777111 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.777596 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.777597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.777597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.777665 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.777933 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.777934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.777934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.778064 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.778103 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.778376 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.778477 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 14:35:50.779271 13874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.770030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:40859 (local address 127.0.0.1:42066)
0811 14:35:50.770292 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:50.770311 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:50.770384 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:50.770920 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:50.770921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:50.771225 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:50.771347 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.771349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.772152 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.772153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.773238 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.773241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.773797 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.773798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.773859 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.774517 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:50.774522 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:50.775270 (+ 748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:50.777232 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.777234 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.777234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.777499 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.777746 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.777747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.777747 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.777843 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.778177 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:50.778178 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:50.778228 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 14:35:50.779015 (+ 787us) client_negotiation.cc:240] Negotiation successful
0811 14:35:50.779152 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":221,"threads_started":1}
I20260811 14:35:50.782315 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.770676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55018 (local address 127.2.103.126:34059)
0811 14:35:50.770730 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.770732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.770745 (+ 13us) server_negotiation.cc:407] Connection header received
0811 14:35:50.770810 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.770811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.770824 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.770897 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.775483 (+ 4586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.776270 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.777414 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.777794 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.777873 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.779289 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.779291 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.779292 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.779301 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.781028 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.781436 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.781437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.781437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.781498 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.781739 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.781741 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.781741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.781877 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.781939 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.782158 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.782227 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 14:35:50.782584 13876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.770041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.0.0.1:55018)
0811 14:35:50.770670 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:50.770684 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:50.770740 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:50.774378 (+ 3638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:50.774379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:50.775252 (+ 873us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:50.775371 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.775374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.776379 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.776381 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.777314 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.777317 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.777880 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.777881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.777988 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.778477 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:50.778483 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:50.779190 (+ 707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:50.781113 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.781114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.781114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.781349 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.781562 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.781563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.781563 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.781636 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.781973 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:50.781974 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:50.782023 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 14:35:50.782122 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 14:35:50.782240 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"spinlock_wait_cycles":10624,"thread_start_us":180,"threads_started":1}
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458949, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.126@KRBTEST.COM for oryx/127.2.103.124@KRBTEST.COM
I20260811 14:35:50.794413 13563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.784672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33113 (local address 127.0.0.1:57864)
0811 14:35:50.784763 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:50.784777 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:50.784830 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:50.785134 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:50.785136 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:50.785304 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:50.785346 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.785348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.786367 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.786368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.786947 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.786949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.787471 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.787472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.787531 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.787957 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:50.787962 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:50.789427 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:50.793197 (+ 3770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.793199 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.793199 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.793444 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.793666 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.793668 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.793668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.793737 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.794055 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:50.794056 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:50.794105 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 14:35:50.794174 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 14:35:50.794298 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 14:35:50.794708 13846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.770438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57862 (local address 127.2.103.124:33113)
0811 14:35:50.770506 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.770508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.770908 (+ 400us) server_negotiation.cc:407] Connection header received
0811 14:35:50.770985 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.770986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.770996 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.771049 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.771540 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.772288 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.773541 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.773837 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.773919 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.792036 (+ 18117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.792037 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.792038 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.792045 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.793529 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.793900 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.793901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.793901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.793944 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.794145 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.794146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.794147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.794272 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.794343 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.794510 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.794590 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 14:35:50.795030 13875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.770395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33113 (local address 127.0.0.1:57862)
0811 14:35:50.770834 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:50.770844 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:50.770929 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:50.771079 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:50.771079 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:50.771349 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:50.771457 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.771459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.772385 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.772386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.773419 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.773421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.773930 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.773931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.774052 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.791239 (+ 17187us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:50.791246 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:50.791913 (+ 667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:50.793613 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.793614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.793615 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.793828 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.794012 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.794013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.794013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.794078 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.794345 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:50.794346 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:50.794393 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 14:35:50.794816 (+ 423us) client_negotiation.cc:240] Negotiation successful
0811 14:35:50.794912 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":171,"threads_started":1}
I20260811 14:35:50.795116 13878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.784710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57864 (local address 127.2.103.124:33113)
0811 14:35:50.784987 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.784989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.784996 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:35:50.785039 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.785039 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.785051 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.785118 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.785443 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.786278 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.787023 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.787376 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.787452 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.789553 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.789555 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.789555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.789562 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.793108 (+ 3546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.793527 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.793528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.793528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.793607 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.793820 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.793821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.793821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.793966 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.794060 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.794668 (+ 608us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.794768 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":196,"threads_started":1}
I20260811 14:35:50.795769 12643 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.2.103.124:33113: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030"
member_type: NON_VOTER
last_known_addr {
host: "127.2.103.124"
port: 33113
}
attrs {
promote: true
}
}
dest_uuid: "4d882bd3213f40e182237f47cbaefda5"
cas_config_opid_index: 5
I20260811 14:35:50.796063 12643 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true } }
I20260811 14:35:50.796448 13564 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER c7e1ff4603b74cfa81ef09f40f4dc030 (127.2.103.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true } } }
I20260811 14:35:50.796749 13801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:50.797571 13564 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.2.103.124:33113
I20260811 14:35:50.797652 13564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4d882bd3213f40e182237f47cbaefda5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true } } }
I20260811 14:35:50.797734 13564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:50.797823 13801 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 1 FOLLOWER]: Refusing update from remote peer 4d882bd3213f40e182237f47cbaefda5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 14:35:50.798097 13564 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:50.798759 13729 master.cc:537] Master@127.2.103.124:33113 shutting down...
I20260811 14:35:50.799420 13801 raft_consensus.cc:858] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true } }
W20260811 14:35:50.799559 13801 raft_consensus.cc:1565] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true } }
I20260811 14:35:50.800374 13801 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2abc19ebb22040029f5122671c37a3d9 (127.2.103.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } } }
I20260811 14:35:50.801385 13563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.796700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:40859 (local address 127.0.0.1:42070)
0811 14:35:50.796919 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:50.796940 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:50.796987 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:50.797274 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:50.797275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:50.797277 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:50.797320 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.797321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.798939 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.798940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.800216 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.800219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.801120 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.801121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.801193 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.801218 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 14:35:50.801273 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 14:35:50.801313 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204}
I20260811 14:35:50.801421 13848 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4d882bd3213f40e182237f47cbaefda5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } } }
I20260811 14:35:50.801486 13848 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 14:35:50.801522 13848 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4d882bd3213f40e182237f47cbaefda5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } } }
I20260811 14:35:50.801554 13848 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 14:35:50.801586 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.796847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42070 (local address 127.2.103.125:40859)
0811 14:35:50.796890 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.796892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.797067 (+ 175us) server_negotiation.cc:407] Connection header received
0811 14:35:50.797100 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.797101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.797142 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.797200 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:50.797393 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.798221 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.800323 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.800940 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.801009 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.801085 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.801484 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.801525 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:35:50.801352 13888 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d882bd3213f40e182237f47cbaefda5. Latest consensus state: current_term: 1 leader_uuid: "4d882bd3213f40e182237f47cbaefda5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } } }
I20260811 14:35:50.801755 13888 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 14:35:50.802022 13564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4d882bd3213f40e182237f47cbaefda5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true } } }
I20260811 14:35:50.802109 13564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:50.802134 13657 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 1 LEARNER]: Refusing update from remote peer 4d882bd3213f40e182237f47cbaefda5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 14:35:50.802148 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 12746
I20260811 14:35:50.808600 13880 mvcc.cc:205] Tried to move back new op lower bound from 7317335858105618432 to 7317335857955758080. Current Snapshot: MvccSnapshot[applied={T|T < 7317335857967095808}]
I20260811 14:35:50.814116 13565 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260811 14:35:50.814463 12620 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 -> Peer c7e1ff4603b74cfa81ef09f40f4dc030 (127.2.103.124:33113): Couldn't send request to peer c7e1ff4603b74cfa81ef09f40f4dc030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 14:35:50.815040 13657 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER c7e1ff4603b74cfa81ef09f40f4dc030 (127.2.103.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true } } }
I20260811 14:35:50.815711 13729 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260811 14:35:50.815845 13729 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260811 14:35:50.815887 13729 tablet_replica.cc:330] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: stopping tablet replica
I20260811 14:35:50.816740 13565 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5: attempting to promote NON_VOTER 2abc19ebb22040029f5122671c37a3d9 to VOTER
I20260811 14:35:50.816742 13702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d882bd3213f40e182237f47cbaefda5. Latest consensus state: current_term: 1 leader_uuid: "4d882bd3213f40e182237f47cbaefda5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true } } }
I20260811 14:35:50.816859 13702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [sys.catalog]: This master's current role is: LEARNER
I20260811 14:35:50.816905 13702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4d882bd3213f40e182237f47cbaefda5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: NON_VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: true } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true } } }
I20260811 14:35:50.816964 13702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [sys.catalog]: This master's current role is: LEARNER
I20260811 14:35:50.817015 13565 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true } }
I20260811 14:35:50.817600 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:45703
--local_ip_for_outbound_sockets=127.2.103.65
--tserver_master_addrs=127.2.103.126:34059
--webserver_port=39897
--webserver_interface=127.2.103.65
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.65.keytab
--principal=oryx/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.103.126:34059,127.2.103.125:40859
--tserver_master_addrs=127.2.103.126:34059,127.2.103.125:40859,127.2.103.124:33113 with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:50.817903 12620 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 -> Peer c7e1ff4603b74cfa81ef09f40f4dc030 (127.2.103.124:33113): Couldn't send request to peer c7e1ff4603b74cfa81ef09f40f4dc030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 14:35:50.818058 13657 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 1 LEARNER]: Refusing update from remote peer 4d882bd3213f40e182237f47cbaefda5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 14:35:50.818542 13564 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 14:35:50.819959 13891 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 2abc19ebb22040029f5122671c37a3d9 (127.2.103.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true } } }
I20260811 14:35:50.821003 13891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4d882bd3213f40e182237f47cbaefda5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true } } }
I20260811 14:35:50.821089 13891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:50.821158 13564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4d882bd3213f40e182237f47cbaefda5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true } } }
I20260811 14:35:50.821213 13564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:50.821462 13657 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 2abc19ebb22040029f5122671c37a3d9 (127.2.103.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true } } }
I20260811 14:35:50.821682 13729 master.cc:552] Master@127.2.103.124:33113 shutdown complete.
I20260811 14:35:50.821720 13729 master_runner.cc:305] Clearing existing system tablet
I20260811 14:35:50.821862 13729 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:50.821906 13729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:50.822491 13890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4d882bd3213f40e182237f47cbaefda5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true } } }
I20260811 14:35:50.822600 13890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:35:50.823020 13729 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 14:35:50.823623 13899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:50.823791 13729 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:50.823853 13729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:50.823877 13729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/wal
uuid: "c7e1ff4603b74cfa81ef09f40f4dc030"
format_stamp: "Formatted at 2026-08-11 14:35:50 on dist-test-slave-twwt"
server_key: "4ebe94b8fc91e1dd39488fc5bbcbe805"
server_key_iv: "7217b1dfdb39855ae54a7b1447906452"
server_key_version: "encryptionkey@0"
I20260811 14:35:50.824697 13729 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:50.826125 13729 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 14:35:50.826201 13729 log.cc:1196] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 14:35:50.826457 13729 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: Deleting consensus metadata
I20260811 14:35:50.826737 13729 master_runner.cc:315] Copying system tablet from 127.2.103.126:34059
I20260811 14:35:50.827474 13729 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: tablet copy: Beginning tablet copy session from remote peer at address 127.2.103.126:34059
I20260811 14:35:50.840647 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.832959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55030 (local address 127.2.103.126:34059)
0811 14:35:50.833020 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.833023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.833298 (+ 275us) server_negotiation.cc:407] Connection header received
0811 14:35:50.833362 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.833364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.833377 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.833449 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.833868 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.834687 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.835813 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.836140 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.836219 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.837436 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.837437 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.837438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.837447 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.839342 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.839771 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.839772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.839773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.839840 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.840062 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.840063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.840064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.840211 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.840286 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.840458 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.840546 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 14:35:50.840660 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.833044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.0.0.1:55030)
0811 14:35:50.833207 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:50.833220 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:50.833301 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:50.833458 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:50.833459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:50.833725 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:50.833791 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.833794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.834768 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.834769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.835715 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.835718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.836227 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.836228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.836316 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.836882 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:50.836887 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:50.837334 (+ 447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:50.839446 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.839447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.839448 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.839695 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.839916 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.839917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.839918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.839991 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.840287 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:50.840289 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:50.840343 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 14:35:50.840414 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 14:35:50.840534 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":99,"threads_started":1}
I20260811 14:35:50.841332 12675 tablet_copy_service.cc:140] P 4d882bd3213f40e182237f47cbaefda5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c7e1ff4603b74cfa81ef09f40f4dc030 ({username='oryx', principal='oryx/127.2.103.124@KRBTEST.COM'} at 127.0.0.1:55030)
I20260811 14:35:50.841405 12675 tablet_copy_service.cc:161] P 4d882bd3213f40e182237f47cbaefda5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c7e1ff4603b74cfa81ef09f40f4dc030 at {username='oryx', principal='oryx/127.2.103.124@KRBTEST.COM'} at 127.0.0.1:55030: session id = c7e1ff4603b74cfa81ef09f40f4dc030-00000000000000000000000000000000
I20260811 14:35:50.842017 12675 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 14:35:50.842682 13729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 14:35:50.844074 13729 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: tablet copy: Starting download of 0 data blocks...
I20260811 14:35:50.844221 13729 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: tablet copy: Starting download of 1 WAL segments...
I20260811 14:35:50.845517 13729 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 14:35:50.846505 12675 tablet_copy_service.cc:342] P 4d882bd3213f40e182237f47cbaefda5: Request end of tablet copy session c7e1ff4603b74cfa81ef09f40f4dc030-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.2.103.124@KRBTEST.COM'} at 127.0.0.1:55030
I20260811 14:35:50.846573 12675 tablet_copy_service.cc:433] P 4d882bd3213f40e182237f47cbaefda5: ending tablet copy session c7e1ff4603b74cfa81ef09f40f4dc030-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c7e1ff4603b74cfa81ef09f40f4dc030
I20260811 14:35:50.848968 13729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:50.850210 13912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:50.851629 13914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:50.851732 13918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:50.851886 13729 server_base.cc:1034] running on GCE node
I20260811 14:35:50.852095 13729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:50.852254 13729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:50.853384 13729 hybrid_clock.cc:648] HybridClock initialized: now 1786458950853363 us; error 37 us; skew 500 ppm
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458950, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:50.854993 13729 init.cc:377] Logged in from keytab as oryx/127.2.103.124@KRBTEST.COM (short username oryx)
I20260811 14:35:50.855278 13729 webserver.cc:460] Webserver started at http://127.2.103.124:45849/ using document root <none> and password file <none>
I20260811 14:35:50.855399 13729 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:50.855433 13729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:50.856196 13729 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:35:50.856832 13928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:50.857019 13729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:50.857074 13729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/wal
uuid: "c7e1ff4603b74cfa81ef09f40f4dc030"
format_stamp: "Formatted at 2026-08-11 14:35:50 on dist-test-slave-twwt"
server_key: "4ebe94b8fc91e1dd39488fc5bbcbe805"
server_key_iv: "7217b1dfdb39855ae54a7b1447906452"
server_key_version: "encryptionkey@0"
I20260811 14:35:50.857138 13729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:50.874245 13729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:50.876287 13729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:50.879743 13729 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.124:33113
I20260811 14:35:50.880230 13729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/master-2/data/info.pb
I20260811 14:35:50.881220 14021 sys_catalog.cc:263] Verifying existing consensus state
I20260811 14:35:50.881794 14021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: Bootstrap starting.
I20260811 14:35:50.882719 14020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.124:33113 every 8 connection(s)
I20260811 14:35:50.884541 14021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:35:50.884789 14021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: Bootstrap complete.
I20260811 14:35:50.885022 14021 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true } }
I20260811 14:35:50.885123 14021 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7e1ff4603b74cfa81ef09f40f4dc030, State: Initialized, Role: LEARNER
I20260811 14:35:50.885166 14021 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true } }
I20260811 14:35:50.885354 14021 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:50.886893 14040 catalog_manager.cc:1259] Loaded cluster ID: 2af9f7732d9744fb80bf2acaaec01923
I20260811 14:35:50.886989 14040 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: loading cluster ID for follower catalog manager: success
I20260811 14:35:50.887985 14024 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true } } }
I20260811 14:35:50.888096 14024 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [sys.catalog]: This master's current role is: LEARNER
I20260811 14:35:50.888511 14040 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: acquiring CA information for follower catalog manager: success
I20260811 14:35:50.888820 14040 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 14:35:50.919999 13893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:50.920225 13893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:50.920251 13893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:50.921452 13893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:50.921495 13893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:50.921522 13893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:35:50.923243 13893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.65.keytab
--principal=oryx/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:45703
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=39897
--tserver_master_addrs=127.2.103.126:34059,127.2.103.125:40859,127.2.103.124:33113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:50.923511 13893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:50.923774 13893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:50.930058 14052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:50.930109 14056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:50.930035 14047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:50.930322 13893 server_base.cc:1034] running on GCE node
I20260811 14:35:50.930528 13893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:50.930814 13893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:50.932036 13893 hybrid_clock.cc:648] HybridClock initialized: now 1786458950932011 us; error 38 us; skew 500 ppm
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458950, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:50.934870 13893 init.cc:377] Logged in from keytab as oryx/127.2.103.65@KRBTEST.COM (short username oryx)
I20260811 14:35:50.935393 13893 webserver.cc:460] Webserver started at http://127.2.103.65:39897/ using document root <none> and password file <none>
I20260811 14:35:50.935662 13893 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:50.935755 13893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:50.937233 13893 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:35:50.937884 14068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:50.938064 13893 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:50.938172 13893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "d6025d4830f2498cb4f8430ac71bc2d0"
format_stamp: "Formatted at 2026-08-11 14:35:49 on dist-test-slave-twwt"
server_key: "a1236778cbe6abccb240a20fca8b6356"
server_key_iv: "4966cb62d98eb294b1e640e560e3f6c4"
server_key_version: "encryptionkey@0"
I20260811 14:35:50.938552 13893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:50.946535 13893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:50.948860 13893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:50.949033 13893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:50.949330 13893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:50.949661 13893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:50.949713 13893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:50.949755 13893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:50.949781 13893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:50.960106 13893 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:45703
I20260811 14:35:50.960151 14251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:45703 every 8 connection(s)
I20260811 14:35:50.960551 13893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:35:50.966514 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 13893
I20260811 14:35:50.966723 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 12985
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458950, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.125@KRBTEST.COM
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458950, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM
Aug 11 14:35:50 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458950, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.124@KRBTEST.COM
I20260811 14:35:50.974376 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.962240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:50651 (local address 127.2.103.125:40859)
0811 14:35:50.962293 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.962295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.962419 (+ 124us) server_negotiation.cc:407] Connection header received
0811 14:35:50.962731 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.962733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.962747 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.962810 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.963814 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.964311 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.965813 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.966116 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.966161 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.970668 (+ 4507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.970669 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.970669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.970677 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.972401 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.973054 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.973056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.973056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.973123 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.973503 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.973504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.973505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.973678 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.973724 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.974183 (+ 459us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.974275 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 14:35:50.974507 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.961566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:35065 (local address 127.2.103.126:34059)
0811 14:35:50.961627 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.961630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.961933 (+ 303us) server_negotiation.cc:407] Connection header received
0811 14:35:50.962765 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.962766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.962781 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.962814 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.963812 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.964591 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.965813 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.966125 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.966162 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.969867 (+ 3705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.969869 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.969870 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.969880 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.972733 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.973294 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.973296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.973296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.973386 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.973642 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.973643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.973644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.973815 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.973869 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.974232 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.974355 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:35:50.974684 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.961951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33113 (local address 127.2.103.65:33513)
0811 14:35:50.962218 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:50.962230 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:50.962643 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:50.963016 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:50.963017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:50.963461 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:50.963694 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.963701 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.964882 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.964883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.965684 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.965686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.966165 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.966166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.966344 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.967357 (+ 1013us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:50.967369 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:50.970220 (+ 2851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:50.972687 (+ 2467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.972688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.972688 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.973053 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.973290 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.973291 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.973292 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.973415 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.973767 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:50.973775 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:50.973965 (+ 190us) client_negotiation.cc:769] Sending connection context
0811 14:35:50.974153 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 14:35:50.974344 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":14,"thread_start_us":88,"threads_started":1}
I20260811 14:35:50.975081 14264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.962162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:40859 (local address 127.2.103.65:50651)
0811 14:35:50.962334 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:50.962343 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:50.962651 (+ 308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:50.962890 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:50.962894 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:50.963486 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:50.963694 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.963699 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.964455 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.964456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.965701 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.965702 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.966248 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.966249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.966369 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.967355 (+ 986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:50.967367 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:50.970528 (+ 3161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:50.972507 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.972517 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.972520 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.972952 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.973200 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.973201 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.973202 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.973413 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.973775 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:50.973776 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:50.973965 (+ 189us) client_negotiation.cc:769] Sending connection context
0811 14:35:50.974835 (+ 870us) client_negotiation.cc:240] Negotiation successful
0811 14:35:50.974976 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":74,"thread_start_us":116,"threads_started":1}
I20260811 14:35:50.975170 14262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.961980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:33513 (local address 127.2.103.124:33113)
0811 14:35:50.962367 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:50.962369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:50.962379 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:35:50.962800 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:50.962801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:50.962814 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:50.962947 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:50.963810 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.964797 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.965799 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.966035 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.966157 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.970328 (+ 4171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:50.970330 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:50.970330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:50.970338 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:50.972604 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.973133 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.973134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.973134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.973199 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:50.973494 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:50.973495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:50.973496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:50.973670 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:50.973736 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:50.974961 (+ 1225us) server_negotiation.cc:299] Negotiation successful
0811 14:35:50.975056 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":59,"threads_started":1}
I20260811 14:35:50.975703 14257 heartbeater.cc:347] Connected to a master server at 127.2.103.125:40859
I20260811 14:35:50.975823 14254 heartbeater.cc:347] Connected to a master server at 127.2.103.124:33113
I20260811 14:35:50.975870 14254 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:50.975831 14257 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:50.976029 14259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:50.961568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.65:35065)
0811 14:35:50.961760 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:50.961798 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:50.962636 (+ 838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:50.962894 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:50.962894 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:50.963431 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:50.963694 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.963701 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.964678 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.964679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.965697 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:50.965700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:50.966249 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:50.966250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:50.966363 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:50.967355 (+ 992us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:50.967367 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:50.969731 (+ 2364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:50.972844 (+ 3113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.972845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.972846 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.973186 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.973478 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:50.973479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:50.973479 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:50.973556 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:50.973925 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:50.973926 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:50.973982 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 14:35:50.974977 (+ 995us) client_negotiation.cc:240] Negotiation successful
0811 14:35:50.975119 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":7,"thread_start_us":106,"threads_started":1}
I20260811 14:35:50.976118 14254 heartbeater.cc:511] Master 127.2.103.124:33113 requested a full tablet report, sending...
I20260811 14:35:50.976118 14257 heartbeater.cc:511] Master 127.2.103.125:40859 requested a full tablet report, sending...
I20260811 14:35:50.976755 13632 ts_manager.cc:194] Registered new tserver with Master: d6025d4830f2498cb4f8430ac71bc2d0 (127.2.103.65:45703)
I20260811 14:35:50.976805 13961 ts_manager.cc:194] Registered new tserver with Master: d6025d4830f2498cb4f8430ac71bc2d0 (127.2.103.65:45703)
I20260811 14:35:50.993460 14255 heartbeater.cc:347] Connected to a master server at 127.2.103.126:34059
I20260811 14:35:50.993599 14255 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:50.993718 14255 heartbeater.cc:511] Master 127.2.103.126:34059 requested a full tablet report, sending...
I20260811 14:35:50.994102 12643 ts_manager.cc:194] Re-registered known tserver with Master: d6025d4830f2498cb4f8430ac71bc2d0 (127.2.103.65:45703)
I20260811 14:35:50.994542 12643 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:35065
I20260811 14:35:50.998180 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:44753
--local_ip_for_outbound_sockets=127.2.103.66
--tserver_master_addrs=127.2.103.126:34059
--webserver_port=46185
--webserver_interface=127.2.103.66
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.66.keytab
--principal=oryx/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.103.126:34059,127.2.103.125:40859
--tserver_master_addrs=127.2.103.126:34059,127.2.103.125:40859,127.2.103.124:33113 with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:51.085117 14265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:51.085402 14265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:51.085449 14265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:51.086716 14265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:51.086759 14265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:51.086787 14265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:35:51.088258 14265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.66.keytab
--principal=oryx/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:44753
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=46185
--tserver_master_addrs=127.2.103.126:34059,127.2.103.125:40859,127.2.103.124:33113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:51.088456 14265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:51.088650 14265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:51.093860 14276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:51.093959 14281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:51.093843 14274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:51.094184 14265 server_base.cc:1034] running on GCE node
I20260811 14:35:51.094436 14265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:51.094738 14265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:51.095916 14265 hybrid_clock.cc:648] HybridClock initialized: now 1786458951095905 us; error 29 us; skew 500 ppm
Aug 11 14:35:51 dist-test-slave-twwt krb5kdc[12573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458951, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:51.099148 14265 init.cc:377] Logged in from keytab as oryx/127.2.103.66@KRBTEST.COM (short username oryx)
I20260811 14:35:51.099658 14265 webserver.cc:460] Webserver started at http://127.2.103.66:46185/ using document root <none> and password file <none>
I20260811 14:35:51.099908 14265 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:51.099988 14265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:51.101425 14265 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 14:35:51.101990 14294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:51.102164 14265 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:51.102267 14265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "3d80c3098fb34a9b97eda8e7427eb3d6"
format_stamp: "Formatted at 2026-08-11 14:35:50 on dist-test-slave-twwt"
server_key: "5d392d48d280500d670b35a4f7e62506"
server_key_iv: "cbd9c2db7b8d6ed302b1e30e76bb37ed"
server_key_version: "encryptionkey@0"
I20260811 14:35:51.102663 14265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:51.123286 14265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:51.125339 14265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:51.125520 14265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:51.125752 14265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:51.126092 14265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:51.126132 14265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:51.126155 14265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:51.126168 14265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:51.134929 14265 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:44753
I20260811 14:35:51.135025 14471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:44753 every 8 connection(s)
I20260811 14:35:51.135401 14265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
Aug 11 14:35:51 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458951, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.124@KRBTEST.COM
Aug 11 14:35:51 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458951, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM
I20260811 14:35:51.144349 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 14265
I20260811 14:35:51.144519 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 13199
Aug 11 14:35:51 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458951, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.125@KRBTEST.COM
I20260811 14:35:51.150063 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.136870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:43781 (local address 127.2.103.125:40859)
0811 14:35:51.136943 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:51.136945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:51.138162 (+ 1217us) server_negotiation.cc:407] Connection header received
0811 14:35:51.138219 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:51.138220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:51.138232 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:51.138284 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:51.139003 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.139793 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.141919 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.142193 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.142230 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.145913 (+ 3683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:51.145915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:51.145916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:51.145926 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:51.148106 (+ 2180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.148750 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.148752 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.148753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.148811 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.149139 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.149140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.149140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.149368 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:51.149422 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:51.149874 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 14:35:51.149968 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 14:35:51.150267 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.136313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33113 (local address 127.2.103.66:50997)
0811 14:35:51.137953 (+ 1640us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:51.137965 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:51.138020 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:51.138234 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:51.138234 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:51.138663 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:51.138913 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.138920 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.139657 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.139658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.140960 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.140963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.141427 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.141428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.141589 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.142340 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:51.142351 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:51.144135 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:51.148675 (+ 4540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:51.148676 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:51.148677 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:51.148917 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:51.149135 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:51.149136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:51.149136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:51.149192 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:51.149621 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:51.149621 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:51.149674 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 14:35:51.149795 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 14:35:51.149992 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1611,"mutex_wait_us":20,"spinlock_wait_cycles":8448,"thread_start_us":206,"threads_started":1}
I20260811 14:35:51.150652 14262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.136362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:50997 (local address 127.2.103.124:33113)
0811 14:35:51.136424 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:51.136426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:51.138046 (+ 1620us) server_negotiation.cc:407] Connection header received
0811 14:35:51.138081 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:51.138083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:51.138101 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:51.138179 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:51.139042 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.139544 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.141066 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.141276 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.141409 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.144262 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:51.144264 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:51.144264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:51.144272 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:51.148581 (+ 4309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.149003 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.149004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.149005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.149064 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.149361 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.149363 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.149363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.149534 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:51.149595 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:51.150410 (+ 815us) server_negotiation.cc:299] Negotiation successful
0811 14:35:51.150512 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 14:35:51.151227 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.136806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:40859 (local address 127.2.103.66:43781)
0811 14:35:51.138101 (+ 1295us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:51.138109 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:51.138162 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:51.138287 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:51.138288 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:51.138594 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:51.138913 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.138918 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.141069 (+ 2151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.141070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.141814 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.141819 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.142335 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.142336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.142415 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.142853 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:51.142858 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:51.145797 (+ 2939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:51.148221 (+ 2424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:51.148232 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:51.148236 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:51.148660 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:51.148896 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:51.148896 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:51.148897 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:51.149051 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:51.149480 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:51.149488 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:51.149673 (+ 185us) client_negotiation.cc:769] Sending connection context
0811 14:35:51.151048 (+ 1375us) client_negotiation.cc:240] Negotiation successful
0811 14:35:51.151141 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1275,"mutex_wait_us":5,"thread_start_us":90,"threads_started":1}
I20260811 14:35:51.151297 14472 heartbeater.cc:347] Connected to a master server at 127.2.103.124:33113
I20260811 14:35:51.151430 14472 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:51.151712 14472 heartbeater.cc:511] Master 127.2.103.124:33113 requested a full tablet report, sending...
I20260811 14:35:51.152347 14475 heartbeater.cc:347] Connected to a master server at 127.2.103.125:40859
I20260811 14:35:51.152345 13961 ts_manager.cc:194] Registered new tserver with Master: 3d80c3098fb34a9b97eda8e7427eb3d6 (127.2.103.66:44753)
I20260811 14:35:51.152405 14475 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:51.152516 14475 heartbeater.cc:511] Master 127.2.103.125:40859 requested a full tablet report, sending...
I20260811 14:35:51.152878 13632 ts_manager.cc:194] Registered new tserver with Master: 3d80c3098fb34a9b97eda8e7427eb3d6 (127.2.103.66:44753)
I20260811 14:35:51.154021 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.136303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:36943 (local address 127.2.103.126:34059)
0811 14:35:51.136469 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:51.136471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:51.136730 (+ 259us) server_negotiation.cc:407] Connection header received
0811 14:35:51.137865 (+ 1135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:51.137867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:51.137881 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:51.137953 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:51.139047 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.139790 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.141474 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.141752 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.141826 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.148819 (+ 6993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:51.148821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:51.148822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:51.148832 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:51.150963 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.153142 (+ 2179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.153144 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.153145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.153219 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.153472 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.153472 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.153472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.153645 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:51.153701 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:51.153892 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 14:35:51.153959 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260811 14:35:51.154170 14482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.136313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.66:36943)
0811 14:35:51.136597 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:51.136646 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:51.137744 (+ 1098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:51.138102 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:51.138108 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:51.138648 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:51.138913 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.138920 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.139874 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.139875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.140960 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.140963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.141826 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.141828 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.141964 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.142515 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:51.142519 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:51.145413 (+ 2894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:51.152596 (+ 7183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:51.152597 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:51.152598 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:51.153062 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:51.153304 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:51.153305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:51.153305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:51.153401 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:51.153715 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:51.153716 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:51.153776 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 14:35:51.153876 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 14:35:51.154009 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":34,"thread_start_us":208,"threads_started":1}
I20260811 14:35:51.154628 14474 heartbeater.cc:347] Connected to a master server at 127.2.103.126:34059
I20260811 14:35:51.154685 14474 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:51.154752 14474 heartbeater.cc:511] Master 127.2.103.126:34059 requested a full tablet report, sending...
I20260811 14:35:51.159814 12643 ts_manager.cc:194] Re-registered known tserver with Master: 3d80c3098fb34a9b97eda8e7427eb3d6 (127.2.103.66:44753)
I20260811 14:35:51.160446 12643 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:36943
I20260811 14:35:51.164623 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:41451
--local_ip_for_outbound_sockets=127.2.103.67
--tserver_master_addrs=127.2.103.126:34059
--webserver_port=38629
--webserver_interface=127.2.103.67
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.67.keytab
--principal=oryx/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.103.126:34059,127.2.103.125:40859
--tserver_master_addrs=127.2.103.126:34059,127.2.103.125:40859,127.2.103.124:33113 with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:51.251361 14484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:51.251613 14484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:51.251657 14484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:51.252991 14484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:51.253055 14484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:51.253101 14484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:35:51.254848 14484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.67.keytab
--principal=oryx/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:41451
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=38629
--tserver_master_addrs=127.2.103.126:34059,127.2.103.125:40859,127.2.103.124:33113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:51.255116 14484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:51.255359 14484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:51.260695 14495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:51.260895 14484 server_base.cc:1034] running on GCE node
W20260811 14:35:51.260727 14493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:51.260711 14498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:51.261233 14484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:51.261518 14484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:51.262674 14484 hybrid_clock.cc:648] HybridClock initialized: now 1786458951262658 us; error 30 us; skew 500 ppm
Aug 11 14:35:51 dist-test-slave-twwt krb5kdc[12573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458951, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:51.265271 14484 init.cc:377] Logged in from keytab as oryx/127.2.103.67@KRBTEST.COM (short username oryx)
I20260811 14:35:51.265746 14484 webserver.cc:460] Webserver started at http://127.2.103.67:38629/ using document root <none> and password file <none>
I20260811 14:35:51.266041 14484 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:51.266134 14484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:51.267609 14484 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:51.268235 14518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:51.268474 14484 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:51.268594 14484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "570caddfa3a84f0682765c59723893dd"
format_stamp: "Formatted at 2026-08-11 14:35:50 on dist-test-slave-twwt"
server_key: "98e919a0399ec0c4dc8ee5994a5af7f9"
server_key_iv: "ceb2631708af3a166e0d7f09653fe517"
server_key_version: "encryptionkey@0"
I20260811 14:35:51.269059 14484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:51.279201 14484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:51.281565 14484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:51.281755 14484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:51.282032 14484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:35:51.282390 14484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:51.282447 14484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:51.282492 14484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:51.282527 14484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:51.291890 14484 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:41451
I20260811 14:35:51.291934 14704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:41451 every 8 connection(s)
I20260811 14:35:51.292372 14484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
Aug 11 14:35:51 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458951, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM
I20260811 14:35:51.300721 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 14484
Aug 11 14:35:51 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458951, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.124@KRBTEST.COM
Aug 11 14:35:51 dist-test-slave-twwt krb5kdc[12573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458951, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.125@KRBTEST.COM
I20260811 14:35:51.304904 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.293239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:39163 (local address 127.2.103.126:34059)
0811 14:35:51.293311 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:51.293313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:51.293621 (+ 308us) server_negotiation.cc:407] Connection header received
0811 14:35:51.294862 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:51.294863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:51.294875 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:51.294926 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:51.295741 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.296463 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.297890 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.298087 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.298142 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.300839 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:51.300841 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:51.300842 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:51.300851 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:51.303114 (+ 2263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.303684 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.303686 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.303686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.303758 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.304091 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.304092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.304092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.304243 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:51.304290 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:51.304725 (+ 435us) server_negotiation.cc:299] Negotiation successful
0811 14:35:51.304809 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 14:35:51.305763 14713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.293242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.67:39163)
0811 14:35:51.293550 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:51.293562 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:51.294667 (+ 1105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:51.294978 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:51.294979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:51.295421 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:51.295635 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.295640 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.296560 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.296560 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.297789 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.297792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.298159 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.298160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.298256 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.298832 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:51.298875 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:51.300712 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:51.303200 (+ 2488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:51.303211 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:51.303215 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:51.303586 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:51.303839 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:51.303839 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:51.303840 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:51.303999 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:51.304340 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:51.304347 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:51.304551 (+ 204us) client_negotiation.cc:769] Sending connection context
0811 14:35:51.305275 (+ 724us) client_negotiation.cc:240] Negotiation successful
0811 14:35:51.305515 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":91,"thread_start_us":114,"threads_started":1}
I20260811 14:35:51.305927 14715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.293578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:40859 (local address 127.2.103.67:52223)
0811 14:35:51.293699 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:51.293710 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:51.294696 (+ 986us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:51.294961 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:51.294961 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:51.295489 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:51.295636 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.295642 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.296609 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.296609 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.297789 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.297792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.298885 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.298887 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.298950 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.299330 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:51.299335 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:51.301732 (+ 2397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:51.304743 (+ 3011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:51.304744 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:51.304744 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:51.304955 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:51.305161 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:51.305162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:51.305162 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:51.305216 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:51.305579 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:51.305581 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:51.305628 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 14:35:51.305705 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 14:35:51.305814 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":66,"thread_start_us":79,"threads_started":1}
I20260811 14:35:51.306231 14706 heartbeater.cc:347] Connected to a master server at 127.2.103.126:34059
I20260811 14:35:51.306332 14706 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:51.306324 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.293763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:52223 (local address 127.2.103.125:40859)
0811 14:35:51.293841 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:51.293843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:51.293852 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:35:51.294803 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:51.294805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:51.294819 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:51.294853 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:51.295753 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.296542 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.298509 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.298805 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.298892 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.301843 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:51.301845 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:51.301845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:51.301854 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:51.304663 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.305037 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.305038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.305039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.305091 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.305288 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.305288 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.305289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.305449 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:51.305529 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:51.306150 (+ 621us) server_negotiation.cc:299] Negotiation successful
0811 14:35:51.306236 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 14:35:51.306557 14706 heartbeater.cc:511] Master 127.2.103.126:34059 requested a full tablet report, sending...
I20260811 14:35:51.306813 14709 heartbeater.cc:347] Connected to a master server at 127.2.103.125:40859
I20260811 14:35:51.306870 14709 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:51.306941 14709 heartbeater.cc:511] Master 127.2.103.125:40859 requested a full tablet report, sending...
I20260811 14:35:51.307013 12643 ts_manager.cc:194] Re-registered known tserver with Master: 570caddfa3a84f0682765c59723893dd (127.2.103.67:41451)
I20260811 14:35:51.307473 12643 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:39163
I20260811 14:35:51.307775 13632 ts_manager.cc:194] Registered new tserver with Master: 570caddfa3a84f0682765c59723893dd (127.2.103.67:41451)
I20260811 14:35:51.308789 14262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.293319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:41663 (local address 127.2.103.124:33113)
0811 14:35:51.293378 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:51.293380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:51.293606 (+ 226us) server_negotiation.cc:407] Connection header received
0811 14:35:51.294774 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:51.294776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:51.294789 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:51.294862 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:51.295754 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.296549 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.298197 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.298380 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.298455 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.301208 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:51.301210 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:51.301210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:51.301218 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:51.304414 (+ 3196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.304829 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.304830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.304830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.304900 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.308181 (+ 3281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.308182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.308182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.308319 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:51.308365 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:51.308607 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 14:35:51.308697 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 14:35:51.309798 14714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.293240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33113 (local address 127.2.103.67:41663)
0811 14:35:51.293480 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:51.293524 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:51.294659 (+ 1135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:51.294953 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:51.294959 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:51.295494 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:51.295636 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.295642 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.296641 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.296642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.297788 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.297791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.298456 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.298457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.298528 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.299104 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:51.299108 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:51.301091 (+ 1983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:51.304502 (+ 3411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:51.304502 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:51.304503 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:51.304740 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:51.305042 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:51.305042 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:51.305043 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:51.305100 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:51.308417 (+ 3317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:51.308417 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:51.308461 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 14:35:51.308916 (+ 455us) client_negotiation.cc:240] Negotiation successful
0811 14:35:51.309046 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":92,"thread_start_us":119,"threads_started":1}
I20260811 14:35:51.310153 14705 heartbeater.cc:347] Connected to a master server at 127.2.103.124:33113
I20260811 14:35:51.310204 14705 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:51.310274 14705 heartbeater.cc:511] Master 127.2.103.124:33113 requested a full tablet report, sending...
I20260811 14:35:51.310547 13961 ts_manager.cc:194] Registered new tserver with Master: 570caddfa3a84f0682765c59723893dd (127.2.103.67:41451)
I20260811 14:35:51.312659 14726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.302136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55034 (local address 127.2.103.126:34059)
0811 14:35:51.302277 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:51.302279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:51.302485 (+ 206us) server_negotiation.cc:407] Connection header received
0811 14:35:51.302632 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:51.302634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:51.302651 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:51.302729 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:51.303541 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.304398 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.305746 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.307201 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.307277 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.309325 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:51.309326 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:51.309327 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:51.309334 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:51.311148 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.311701 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.311703 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.311703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.311775 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.312026 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.312028 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.312028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.312184 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:51.312279 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:51.312433 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 14:35:51.312535 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260811 14:35:51.315730 14736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.302480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42078 (local address 127.2.103.125:40859)
0811 14:35:51.304588 (+ 2108us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:51.304590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:51.304598 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:35:51.304619 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:51.304620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:51.304631 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:51.304696 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:51.307598 (+ 2902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.308287 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.310113 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.310555 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.310741 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.312323 (+ 1582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:51.312324 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:51.312324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:51.312332 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:51.314515 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.314934 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.314935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.314935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.315001 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.315194 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.315195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.315195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.315329 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:51.315438 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:51.315546 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 14:35:51.315625 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2084,"thread_start_us":60,"threads_started":1}
I20260811 14:35:51.316169 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.302545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57872 (local address 127.2.103.124:33113)
0811 14:35:51.302829 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:51.302831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:51.303057 (+ 226us) server_negotiation.cc:407] Connection header received
0811 14:35:51.303106 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:51.303108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:51.303121 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:51.303183 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:51.307330 (+ 4147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.308045 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.309982 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.310349 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.310411 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.312159 (+ 1748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:51.312160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:51.312161 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:51.312167 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:51.314586 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.315012 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.315013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.315013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.315069 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.315263 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.315265 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.315265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.315406 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:51.315499 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:51.315985 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 14:35:51.316075 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":97,"threads_started":1}
I20260811 14:35:51.314301 12643 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:35:51.316672 12643 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:35:51.326330 14758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.321656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:45703 (local address 127.0.0.1:33384)
0811 14:35:51.322023 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:51.322033 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:51.322084 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:51.322496 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:51.322497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:51.322499 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:51.322553 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.322555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.324244 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.324245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.325078 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.325080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.326065 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.326066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.326129 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.326133 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:35:51.326185 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 14:35:51.326226 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":1,"thread_start_us":57,"threads_started":1}
I20260811 14:35:51.326593 14751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.321797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:44753 (local address 127.0.0.1:34100)
0811 14:35:51.322153 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:51.322161 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:51.322204 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:51.322456 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:51.322458 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:51.322461 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:51.322558 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.322580 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.323841 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.323842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.325011 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.325014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.326141 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.326142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.326210 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.326216 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 14:35:51.326265 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 14:35:51.326308 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":75,"threads_started":1}
I20260811 14:35:51.326830 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.321920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34100 (local address 127.2.103.66:44753)
0811 14:35:51.322133 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:51.322139 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:51.322260 (+ 121us) server_negotiation.cc:407] Connection header received
0811 14:35:51.322311 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:51.322313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:51.322330 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:51.322451 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:51.322988 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.323741 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.325127 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.325930 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.326002 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.326102 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:51.326374 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 14:35:51.326467 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":91,"threads_started":1}
I20260811 14:35:51.326911 14760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.322047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33384 (local address 127.2.103.65:45703)
0811 14:35:51.322257 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:51.322261 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:51.322271 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:35:51.322311 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:51.322313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:51.322330 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:51.322414 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:51.322694 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.324133 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.325167 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.325856 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.325927 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.326025 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:51.326745 (+ 720us) server_negotiation.cc:299] Negotiation successful
0811 14:35:51.326803 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":168,"threads_started":1}
I20260811 14:35:51.327400 14750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.321466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:41451 (local address 127.0.0.1:37708)
0811 14:35:51.321658 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:51.321671 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:51.321722 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:51.322793 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:51.322794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:51.322797 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:51.322830 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.322832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.324228 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.324229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.325341 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.325343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.327200 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.327201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.327267 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.327271 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:35:51.327303 (+ 32us) client_negotiation.cc:240] Negotiation successful
0811 14:35:51.327333 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":29,"thread_start_us":83,"threads_started":1}
I20260811 14:35:51.327556 14361 tablet_service.cc:1467] Processing CreateTablet for tablet 2f48db10817f40fc975af17db12ca7a6 (DEFAULT_TABLE table=test-table [id=eeac5a2a98fc425d9e8caa5e8be51ac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:51.327847 14756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.321632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37708 (local address 127.2.103.67:41451)
0811 14:35:51.322585 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:51.322590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:51.322600 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:35:51.322666 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:51.322668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:51.322685 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:51.322908 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:51.322983 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.324125 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.325423 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.326877 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.327590 (+ 713us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.327688 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:51.327710 (+ 22us) server_negotiation.cc:299] Negotiation successful
0811 14:35:51.327763 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":913,"thread_start_us":167,"threads_started":1}
I20260811 14:35:51.327991 14361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f48db10817f40fc975af17db12ca7a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:51.328540 14596 tablet_service.cc:1467] Processing CreateTablet for tablet 2f48db10817f40fc975af17db12ca7a6 (DEFAULT_TABLE table=test-table [id=eeac5a2a98fc425d9e8caa5e8be51ac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:51.328871 14596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f48db10817f40fc975af17db12ca7a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:51.329806 14762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.325661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33113 (local address 127.0.0.1:57884)
0811 14:35:51.326104 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:51.326115 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:51.326191 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:51.326415 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:51.326416 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:51.326418 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:51.326454 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.326455 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.327403 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.327404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.328631 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.328634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.329568 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.329569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.329630 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.329634 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:35:51.329692 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 14:35:51.329732 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":84,"threads_started":1}
I20260811 14:35:51.329849 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.325844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57884 (local address 127.2.103.124:33113)
0811 14:35:51.325888 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:51.325890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:51.326166 (+ 276us) server_negotiation.cc:407] Connection header received
0811 14:35:51.326271 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:51.326272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:51.326296 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:51.326335 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:51.326534 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.327287 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.328738 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.329340 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.329404 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.329479 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:51.329725 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 14:35:51.329776 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:35:51.331152 14155 tablet_service.cc:1467] Processing CreateTablet for tablet 2f48db10817f40fc975af17db12ca7a6 (DEFAULT_TABLE table=test-table [id=eeac5a2a98fc425d9e8caa5e8be51ac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:51.331549 14155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f48db10817f40fc975af17db12ca7a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:51.332110 14764 tablet_bootstrap.cc:492] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6: Bootstrap starting.
I20260811 14:35:51.332463 14765 tablet_bootstrap.cc:492] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd: Bootstrap starting.
I20260811 14:35:51.333061 14764 tablet_bootstrap.cc:654] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:51.333380 14765 tablet_bootstrap.cc:654] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:51.333405 14764 log.cc:824] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:51.333775 14765 log.cc:824] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:51.334268 14764 tablet_bootstrap.cc:492] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6: No bootstrap required, opened a new log
I20260811 14:35:51.334379 14764 ts_tablet_manager.cc:1397] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:35:51.334690 14765 tablet_bootstrap.cc:492] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd: No bootstrap required, opened a new log
I20260811 14:35:51.334764 14765 ts_tablet_manager.cc:1397] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:35:51.335156 13977 raft_consensus.cc:1203] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260811 14:35:51.335574 14024 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d882bd3213f40e182237f47cbaefda5. Latest consensus state: current_term: 1 leader_uuid: "4d882bd3213f40e182237f47cbaefda5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true } } }
I20260811 14:35:51.335691 14024 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [sys.catalog]: This master's current role is: LEARNER
I20260811 14:35:51.336335 14772 tablet_bootstrap.cc:492] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0: Bootstrap starting.
I20260811 14:35:51.336783 14764 raft_consensus.cc:348] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 } } peers { permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 } } peers { permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 } }
I20260811 14:35:51.336937 14764 raft_consensus.cc:374] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:51.336980 14764 raft_consensus.cc:729] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d80c3098fb34a9b97eda8e7427eb3d6, State: Initialized, Role: FOLLOWER
I20260811 14:35:51.337100 14764 consensus_queue.cc:260] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 } } peers { permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 } } peers { permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 } }
I20260811 14:35:51.337216 14772 tablet_bootstrap.cc:654] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:51.337389 14764 ts_tablet_manager.cc:1428] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:35:51.337524 14474 heartbeater.cc:503] Master 127.2.103.126:34059 was elected leader, sending a full tablet report...
I20260811 14:35:51.337594 14772 log.cc:824] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:51.338205 14765 raft_consensus.cc:348] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 } } peers { permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 } } peers { permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 } }
I20260811 14:35:51.338594 14765 raft_consensus.cc:374] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:51.338792 14765 raft_consensus.cc:729] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 570caddfa3a84f0682765c59723893dd, State: Initialized, Role: FOLLOWER
I20260811 14:35:51.339046 14765 consensus_queue.cc:260] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 } } peers { permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 } } peers { permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 } }
I20260811 14:35:51.339337 14772 tablet_bootstrap.cc:492] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0: No bootstrap required, opened a new log
I20260811 14:35:51.339421 14772 ts_tablet_manager.cc:1397] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 14:35:51.339424 14765 ts_tablet_manager.cc:1428] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 14:35:51.341245 14772 raft_consensus.cc:348] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 } } peers { permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 } } peers { permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 } }
I20260811 14:35:51.341365 14772 raft_consensus.cc:374] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:51.341416 14772 raft_consensus.cc:729] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6025d4830f2498cb4f8430ac71bc2d0, State: Initialized, Role: FOLLOWER
I20260811 14:35:51.341539 14772 consensus_queue.cc:260] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 } } peers { permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 } } peers { permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 } }
I20260811 14:35:51.341782 14772 ts_tablet_manager.cc:1428] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 14:35:51.341856 14255 heartbeater.cc:503] Master 127.2.103.126:34059 was elected leader, sending a full tablet report...
I20260811 14:35:51.341888 14706 heartbeater.cc:503] Master 127.2.103.126:34059 was elected leader, sending a full tablet report...
I20260811 14:35:51.353150 14783 raft_consensus.cc:482] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:35:51.353251 14783 raft_consensus.cc:504] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 } } peers { permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 } } peers { permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 } }
I20260811 14:35:51.353693 14783 leader_election.cc:288] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 570caddfa3a84f0682765c59723893dd (127.2.103.67:41451), 3d80c3098fb34a9b97eda8e7427eb3d6 (127.2.103.66:44753)
I20260811 14:35:51.356670 14264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.353960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:44753 (local address 127.2.103.65:38963)
0811 14:35:51.354016 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:51.354028 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:51.354079 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:51.354274 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:51.354275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:51.354276 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:51.354361 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.354362 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.355136 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.355137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.355853 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.355855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.356400 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.356401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.356494 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.356497 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:35:51.356539 (+ 42us) client_negotiation.cc:240] Negotiation successful
0811 14:35:51.356573 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 14:35:51.356680 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.354100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:38963 (local address 127.2.103.66:44753)
0811 14:35:51.354148 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:51.354150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:51.354158 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:35:51.354184 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:51.354184 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:51.354195 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:51.354253 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:51.354453 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.355047 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.355947 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.356300 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.356396 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.356405 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:51.356563 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 14:35:51.356614 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 14:35:51.356750 14756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.353917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:60721 (local address 127.2.103.67:41451)
0811 14:35:51.353991 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:51.353993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:51.354005 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:35:51.354034 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:51.354035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:51.354046 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:51.354100 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:51.354313 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.354985 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.355945 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.356299 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.356385 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.356396 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:51.356636 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 14:35:51.356675 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 14:35:51.356914 14259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.353847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:41451 (local address 127.2.103.65:60721)
0811 14:35:51.353901 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:51.353914 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:51.353958 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:51.354107 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:51.354109 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:51.354111 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:51.354225 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.354228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.355066 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.355067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.355853 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:51.355855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.356384 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.356385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:51.356553 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.356557 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:35:51.356619 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 14:35:51.356658 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":24}
I20260811 14:35:51.357292 14393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f48db10817f40fc975af17db12ca7a6" candidate_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" is_pre_election: true
I20260811 14:35:51.357304 14633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f48db10817f40fc975af17db12ca7a6" candidate_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "570caddfa3a84f0682765c59723893dd" is_pre_election: true
I20260811 14:35:51.357458 14393 raft_consensus.cc:2454] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6025d4830f2498cb4f8430ac71bc2d0 in term 0.
I20260811 14:35:51.357470 14633 raft_consensus.cc:2454] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6025d4830f2498cb4f8430ac71bc2d0 in term 0.
I20260811 14:35:51.357694 14070 leader_election.cc:302] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d80c3098fb34a9b97eda8e7427eb3d6, d6025d4830f2498cb4f8430ac71bc2d0; no voters:
I20260811 14:35:51.357888 14783 raft_consensus.cc:2781] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:35:51.357951 14783 raft_consensus.cc:482] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:35:51.357969 14783 raft_consensus.cc:3037] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:51.358842 14783 raft_consensus.cc:504] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 } } peers { permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 } } peers { permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 } }
I20260811 14:35:51.359009 14783 leader_election.cc:288] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [CANDIDATE]: Term 1 election: Requested vote from peers 570caddfa3a84f0682765c59723893dd (127.2.103.67:41451), 3d80c3098fb34a9b97eda8e7427eb3d6 (127.2.103.66:44753)
I20260811 14:35:51.359185 14633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f48db10817f40fc975af17db12ca7a6" candidate_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "570caddfa3a84f0682765c59723893dd"
I20260811 14:35:51.359233 14393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f48db10817f40fc975af17db12ca7a6" candidate_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6"
I20260811 14:35:51.359277 14633 raft_consensus.cc:3037] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:51.359330 14393 raft_consensus.cc:3037] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:51.360172 14393 raft_consensus.cc:2454] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6025d4830f2498cb4f8430ac71bc2d0 in term 1.
I20260811 14:35:51.360350 14070 leader_election.cc:302] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d80c3098fb34a9b97eda8e7427eb3d6, d6025d4830f2498cb4f8430ac71bc2d0; no voters:
I20260811 14:35:51.360421 14633 raft_consensus.cc:2454] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6025d4830f2498cb4f8430ac71bc2d0 in term 1.
I20260811 14:35:51.360450 14783 raft_consensus.cc:2781] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:51.360702 14783 raft_consensus.cc:686] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [term 1 LEADER]: Becoming Leader. State: Replica: d6025d4830f2498cb4f8430ac71bc2d0, State: Running, Role: LEADER
I20260811 14:35:51.360813 14783 consensus_queue.cc:237] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 } } peers { permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 } } peers { permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 } }
I20260811 14:35:51.361778 12643 catalog_manager.cc:5495] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6025d4830f2498cb4f8430ac71bc2d0 (127.2.103.65). New cstate: current_term: 1 leader_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 } health_report { overall_health: UNKNOWN } } }
I20260811 14:35:51.362396 13565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4d882bd3213f40e182237f47cbaefda5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: NON_VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: true } } }
I20260811 14:35:51.362507 13565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:51.363036 13564 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5: attempting to promote NON_VOTER c7e1ff4603b74cfa81ef09f40f4dc030 to VOTER
I20260811 14:35:51.363178 13564 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false } }
I20260811 14:35:51.363583 13657 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 1 FOLLOWER]: Refusing update from remote peer 4d882bd3213f40e182237f47cbaefda5: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260811 14:35:51.363621 13977 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 1 LEARNER]: Refusing update from remote peer 4d882bd3213f40e182237f47cbaefda5: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260811 14:35:51.363801 13891 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260811 14:35:51.363922 13564 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260811 14:35:51.364507 13891 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, c7e1ff4603b74cfa81ef09f40f4dc030 (127.2.103.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false } } }
I20260811 14:35:51.364647 13977 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, c7e1ff4603b74cfa81ef09f40f4dc030 (127.2.103.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false } } }
I20260811 14:35:51.365691 13891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4d882bd3213f40e182237f47cbaefda5" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false } } }
I20260811 14:35:51.365727 13564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4d882bd3213f40e182237f47cbaefda5" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false } } }
I20260811 14:35:51.365789 13891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:51.365806 13564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:51.365777 14024 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4d882bd3213f40e182237f47cbaefda5" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false } } }
I20260811 14:35:51.365865 14024 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:35:51.366062 13565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4d882bd3213f40e182237f47cbaefda5" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false } } }
I20260811 14:35:51.366129 13565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:51.366230 13657 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, c7e1ff4603b74cfa81ef09f40f4dc030 (127.2.103.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false } } }
I20260811 14:35:51.367210 14792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4d882bd3213f40e182237f47cbaefda5" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false } } }
I20260811 14:35:51.367311 14792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:35:51.377123 14760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.374855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33392 (local address 127.2.103.65:45703)
0811 14:35:51.374913 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:51.374915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:51.374923 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:35:51.374960 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:51.374961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:51.374979 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:51.375046 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:51.375167 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.375700 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.376446 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.376634 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.376683 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.376807 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:51.376917 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:51.376978 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:51.377007 (+ 29us) server_negotiation.cc:299] Negotiation successful
0811 14:35:51.377025 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 14:35:51.379086 14633 raft_consensus.cc:1261] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd [term 1 FOLLOWER]: Refusing update from remote peer d6025d4830f2498cb4f8430ac71bc2d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:51.379101 14393 raft_consensus.cc:1261] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 1 FOLLOWER]: Refusing update from remote peer d6025d4830f2498cb4f8430ac71bc2d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:51.379464 14786 consensus_queue.cc:1035] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:51.379597 14783 consensus_queue.cc:1035] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:51.383899 14794 mvcc.cc:205] Tried to move back new op lower bound from 7317335864845541376 to 7317335864774504448. Current Snapshot: MvccSnapshot[applied={T|T < 7317335864845541376}]
I20260811 14:35:51.384071 14797 mvcc.cc:205] Tried to move back new op lower bound from 7317335864845541376 to 7317335864774504448. Current Snapshot: MvccSnapshot[applied={T|T < 7317335864845541376}]
I20260811 14:35:51.386796 12640 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:51.386909 12640 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:51.388415 12640 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d882bd3213f40e182237f47cbaefda5: Sending DeleteTablet for 3 replicas of tablet 2f48db10817f40fc975af17db12ca7a6
I20260811 14:35:51.389117 14155 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f48db10817f40fc975af17db12ca7a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:51 UTC) from {username='oryx', principal='oryx/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:33384
I20260811 14:35:51.389370 14596 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f48db10817f40fc975af17db12ca7a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:51 UTC) from {username='oryx', principal='oryx/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:37708
I20260811 14:35:51.389633 14812 tablet_replica.cc:330] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0: stopping tablet replica
I20260811 14:35:51.389748 14812 raft_consensus.cc:2229] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:35:51.389796 14813 tablet_replica.cc:330] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd: stopping tablet replica
I20260811 14:35:51.389887 14813 raft_consensus.cc:2229] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:51.389899 14812 raft_consensus.cc:2258] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:51.389994 14813 raft_consensus.cc:2258] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:51.390395 14361 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f48db10817f40fc975af17db12ca7a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:51 UTC) from {username='oryx', principal='oryx/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:34100
I20260811 14:35:51.390429 14813 ts_tablet_manager.cc:1905] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:51.390429 14812 ts_tablet_manager.cc:1905] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:51.390719 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 12592
I20260811 14:35:51.401780 14813 ts_tablet_manager.cc:1918] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:35:51.401895 14813 log.cc:1196] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/2f48db10817f40fc975af17db12ca7a6
I20260811 14:35:51.402284 14813 ts_tablet_manager.cc:1939] T 2f48db10817f40fc975af17db12ca7a6 P 570caddfa3a84f0682765c59723893dd: Deleting consensus metadata
I20260811 14:35:51.402318 14812 ts_tablet_manager.cc:1918] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:35:51.402396 14812 log.cc:1196] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-0/wal/wals/2f48db10817f40fc975af17db12ca7a6
I20260811 14:35:51.402702 14812 ts_tablet_manager.cc:1939] T 2f48db10817f40fc975af17db12ca7a6 P d6025d4830f2498cb4f8430ac71bc2d0: Deleting consensus metadata
I20260811 14:35:51.403182 14814 tablet_replica.cc:330] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6: stopping tablet replica
I20260811 14:35:51.403296 14814 raft_consensus.cc:2229] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:51.403419 14814 raft_consensus.cc:2258] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:51.403976 14814 ts_tablet_manager.cc:1905] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:51.405411 14814 ts_tablet_manager.cc:1918] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:35:51.405506 14814 log.cc:1196] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/2f48db10817f40fc975af17db12ca7a6
I20260811 14:35:51.405781 14814 ts_tablet_manager.cc:1939] T 2f48db10817f40fc975af17db12ca7a6 P 3d80c3098fb34a9b97eda8e7427eb3d6: Deleting consensus metadata
W20260811 14:35:51.407217 14296 connection.cc:535] server connection from 127.0.0.1:34100 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 14:35:51.407475 14072 connection.cc:535] server connection from 127.0.0.1:33384 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 14:35:51.407691 14520 connection.cc:535] server connection from 127.0.0.1:37708 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 14:35:51.416122 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.389692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57886 (local address 127.2.103.124:33113)
0811 14:35:51.389735 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:51.389738 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:51.406822 (+ 17084us) server_negotiation.cc:407] Connection header received
0811 14:35:51.406912 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:51.406913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:51.406926 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:51.406993 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:51.407869 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.408570 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.409648 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.409978 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.410023 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.412269 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:51.412271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:51.412271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:51.412279 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:51.414475 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.415064 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.415065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.415066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.415161 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.415398 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.415400 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.415400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.415557 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:51.415685 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:51.415917 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 14:35:51.416000 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:35:51.418100 14262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.409884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57896 (local address 127.2.103.124:33113)
0811 14:35:51.409986 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:51.409988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:51.410092 (+ 104us) server_negotiation.cc:407] Connection header received
0811 14:35:51.410238 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:51.410239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:51.410250 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:51.410280 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:51.411095 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.411791 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.412901 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.413187 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.413228 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.415025 (+ 1797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:51.415026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:51.415026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:51.415033 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:51.416988 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.417318 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.417319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.417319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.417381 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.417584 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.417585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.417585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.417695 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:51.417792 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:51.417906 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 14:35:51.417981 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 14:35:51.419423 14736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:51.409834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42082 (local address 127.2.103.125:40859)
0811 14:35:51.409932 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:51.409934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:51.409943 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:35:51.409997 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:51.409998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:51.410014 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:51.410047 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:51.410977 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.411739 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.412912 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:51.413221 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:51.413261 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:51.414982 (+ 1721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:51.414984 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:51.414984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:51.414993 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:51.418103 (+ 3110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.418567 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.418568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.418568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.418716 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:51.418905 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:51.418906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:51.418906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:51.419044 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:51.419142 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:51.419233 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 14:35:51.419322 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 14:35:52.385183 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:52.384804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.65:53965)
0811 14:35:52.384987 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:52.385078 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":97,"threads_started":1}
I20260811 14:35:52.385409 14835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:52.385037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.66:34193)
0811 14:35:52.385271 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:52.385359 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":60,"threads_started":1}
W20260811 14:35:52.385428 14255 heartbeater.cc:644] Failed to heartbeat to 127.2.103.126:34059 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
I20260811 14:35:52.385449 14833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:52.384861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.67:36287)
0811 14:35:52.385285 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:52.385354 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":54,"threads_started":1}
W20260811 14:35:52.385636 14706 heartbeater.cc:644] Failed to heartbeat to 127.2.103.126:34059 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
W20260811 14:35:52.385764 14474 heartbeater.cc:644] Failed to heartbeat to 127.2.103.126:34059 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
I20260811 14:35:52.889490 14837 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4d882bd3213f40e182237f47cbaefda5)
I20260811 14:35:52.889688 14837 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false } }
I20260811 14:35:52.889685 14839 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4d882bd3213f40e182237f47cbaefda5)
I20260811 14:35:52.889863 14839 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false } }
I20260811 14:35:52.890008 14837 leader_election.cc:288] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4d882bd3213f40e182237f47cbaefda5 (127.2.103.126:34059), c7e1ff4603b74cfa81ef09f40f4dc030 (127.2.103.124:33113)
I20260811 14:35:52.890424 14840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:52.890168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.0.0.1:40768)
0811 14:35:52.890317 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:52.890374 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
W20260811 14:35:52.890543 13610 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
I20260811 14:35:52.890489 14844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:52.890234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.0.0.1:40772)
0811 14:35:52.890385 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:52.890438 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":56,"threads_started":1}
W20260811 14:35:52.890934 13931 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
I20260811 14:35:52.890456 14839 leader_election.cc:288] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4d882bd3213f40e182237f47cbaefda5 (127.2.103.126:34059), 2abc19ebb22040029f5122671c37a3d9 (127.2.103.125:40859)
I20260811 14:35:52.891206 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:52.890939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.0.0.1:40786)
0811 14:35:52.891132 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:52.891176 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172}
W20260811 14:35:52.891299 13610 leader_election.cc:334] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4d882bd3213f40e182237f47cbaefda5 (127.2.103.126:34059): Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
I20260811 14:35:52.891400 14844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:52.891312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.0.0.1:40794)
0811 14:35:52.891353 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:52.891375 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260811 14:35:52.891482 13931 leader_election.cc:334] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4d882bd3213f40e182237f47cbaefda5 (127.2.103.126:34059): Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
I20260811 14:35:52.893541 14841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:52.890278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:40859 (local address 127.0.0.1:40950)
0811 14:35:52.890556 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:52.890567 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:52.890655 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:52.890881 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:52.890882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:52.890885 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:52.890954 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:52.890957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:52.891861 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:52.891862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:52.892729 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:52.892731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:52.893283 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:52.893284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:52.893357 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:52.893361 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:35:52.893428 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 14:35:52.893475 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":44,"threads_started":1}
I20260811 14:35:52.893539 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:52.890399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40950 (local address 127.2.103.125:40859)
0811 14:35:52.890704 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:52.890707 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:52.890714 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:35:52.890752 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:52.890753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:52.890764 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:52.891035 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:52.891078 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:52.891779 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:52.892822 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:52.893198 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:52.893281 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:52.893293 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:52.893433 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 14:35:52.893475 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":33,"threads_started":1}
I20260811 14:35:52.893854 14840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:52.890228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33113 (local address 127.0.0.1:36288)
0811 14:35:52.890495 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:52.890506 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:52.890569 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:52.891158 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:52.891159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:52.891162 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:52.891215 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:52.891216 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:52.892179 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:52.892180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:52.892996 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:52.892999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:52.893582 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:52.893583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:52.893647 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:52.893650 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:35:52.893735 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 14:35:52.893779 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":57,"threads_started":1}
I20260811 14:35:52.893940 14848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:52.890442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36288 (local address 127.2.103.124:33113)
0811 14:35:52.890722 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:52.890724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:52.890731 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:35:52.890757 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:52.890758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:52.890768 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:52.891401 (+ 633us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:52.891445 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:52.892107 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:52.893104 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:52.893485 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:52.893577 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:52.893588 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:52.893830 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 14:35:52.893878 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":51,"threads_started":1}
I20260811 14:35:52.893954 13657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2abc19ebb22040029f5122671c37a3d9" is_pre_election: true
I20260811 14:35:52.894088 13657 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7e1ff4603b74cfa81ef09f40f4dc030 in term 1.
I20260811 14:35:52.894193 13977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2abc19ebb22040029f5122671c37a3d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" is_pre_election: true
I20260811 14:35:52.894256 13929 leader_election.cc:302] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2abc19ebb22040029f5122671c37a3d9, c7e1ff4603b74cfa81ef09f40f4dc030; no voters: 4d882bd3213f40e182237f47cbaefda5
I20260811 14:35:52.894313 13977 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2abc19ebb22040029f5122671c37a3d9 in term 1.
I20260811 14:35:52.894389 14839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 14:35:52.894431 14839 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4d882bd3213f40e182237f47cbaefda5)
I20260811 14:35:52.894454 14839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 1 FOLLOWER]: Advancing to term 2
I20260811 14:35:52.894516 13609 leader_election.cc:302] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2abc19ebb22040029f5122671c37a3d9, c7e1ff4603b74cfa81ef09f40f4dc030; no voters: 4d882bd3213f40e182237f47cbaefda5
I20260811 14:35:52.894673 14837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 14:35:52.894738 14837 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4d882bd3213f40e182237f47cbaefda5)
I20260811 14:35:52.894774 14837 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 1 FOLLOWER]: Advancing to term 2
I20260811 14:35:52.895284 14839 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false } }
I20260811 14:35:52.895427 14839 leader_election.cc:288] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [CANDIDATE]: Term 2 election: Requested vote from peers 4d882bd3213f40e182237f47cbaefda5 (127.2.103.126:34059), 2abc19ebb22040029f5122671c37a3d9 (127.2.103.125:40859)
I20260811 14:35:52.895489 14837 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false } }
I20260811 14:35:52.895593 14841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:52.895501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.0.0.1:40796)
0811 14:35:52.895546 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:52.895569 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260811 14:35:52.895609 13657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2abc19ebb22040029f5122671c37a3d9"
I20260811 14:35:52.895700 13657 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c7e1ff4603b74cfa81ef09f40f4dc030 in current term 2: Already voted for candidate 2abc19ebb22040029f5122671c37a3d9 in this term.
I20260811 14:35:52.895622 14837 leader_election.cc:288] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [CANDIDATE]: Term 2 election: Requested vote from peers 4d882bd3213f40e182237f47cbaefda5 (127.2.103.126:34059), c7e1ff4603b74cfa81ef09f40f4dc030 (127.2.103.124:33113)
I20260811 14:35:52.895814 14840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:52.895695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.0.0.1:40802)
0811 14:35:52.895734 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:52.895763 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260811 14:35:52.895793 13977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2abc19ebb22040029f5122671c37a3d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030"
I20260811 14:35:52.895870 13977 raft_consensus.cc:2379] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2abc19ebb22040029f5122671c37a3d9 in current term 2: Already voted for candidate c7e1ff4603b74cfa81ef09f40f4dc030 in this term.
I20260811 14:35:52.896267 14840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:52.896166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.0.0.1:40812)
0811 14:35:52.896217 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:52.896243 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 14:35:52.896334 13610 leader_election.cc:334] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4d882bd3213f40e182237f47cbaefda5 (127.2.103.126:34059): Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
I20260811 14:35:52.896404 13610 leader_election.cc:302] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2abc19ebb22040029f5122671c37a3d9; no voters: 4d882bd3213f40e182237f47cbaefda5, c7e1ff4603b74cfa81ef09f40f4dc030
I20260811 14:35:52.896410 14841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:52.896315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.0.0.1:40826)
0811 14:35:52.896359 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:52.896384 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260811 14:35:52.896458 13931 leader_election.cc:334] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4d882bd3213f40e182237f47cbaefda5 (127.2.103.126:34059): Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
I20260811 14:35:52.896492 13931 leader_election.cc:302] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7e1ff4603b74cfa81ef09f40f4dc030; no voters: 2abc19ebb22040029f5122671c37a3d9, 4d882bd3213f40e182237f47cbaefda5
I20260811 14:35:52.896492 14837 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 14:35:52.896569 14839 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 14:35:53.386162 14852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:53.385864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.65:37439)
0811 14:35:53.386048 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:53.386095 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":106,"threads_started":1}
I20260811 14:35:53.386206 14857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:53.385990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.66:35917)
0811 14:35:53.386121 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:53.386157 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
I20260811 14:35:53.386207 14854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:53.385975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.67:55323)
0811 14:35:53.386144 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:53.386172 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":76,"threads_started":1}
I20260811 14:35:54.386963 14862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.386665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.67:38311)
0811 14:35:54.386871 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.386910 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":69,"threads_started":1}
I20260811 14:35:54.386963 14865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.386758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.66:60241)
0811 14:35:54.386877 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.386910 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260811 14:35:54.386972 14859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.386599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.65:36441)
0811 14:35:54.386835 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.386913 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":69,"threads_started":1}
I20260811 14:35:54.584230 14867 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:35:54.584359 14867 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false } }
I20260811 14:35:54.584583 14867 leader_election.cc:288] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4d882bd3213f40e182237f47cbaefda5 (127.2.103.126:34059), 2abc19ebb22040029f5122671c37a3d9 (127.2.103.125:40859)
I20260811 14:35:54.584847 13657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2abc19ebb22040029f5122671c37a3d9" is_pre_election: true
I20260811 14:35:54.584985 13657 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7e1ff4603b74cfa81ef09f40f4dc030 in term 2.
I20260811 14:35:54.584983 14869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.584727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.0.0.1:41096)
0811 14:35:54.584893 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.584935 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
I20260811 14:35:54.585186 13929 leader_election.cc:302] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2abc19ebb22040029f5122671c37a3d9, c7e1ff4603b74cfa81ef09f40f4dc030; no voters:
I20260811 14:35:54.585314 14867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 14:35:54.585352 14867 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:35:54.585381 14867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 2 FOLLOWER]: Advancing to term 3
I20260811 14:35:54.585494 14869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.585365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.0.0.1:41102)
0811 14:35:54.585417 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.585460 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 14:35:54.585602 13931 leader_election.cc:334] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4d882bd3213f40e182237f47cbaefda5 (127.2.103.126:34059): Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
I20260811 14:35:54.586352 14867 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false } }
I20260811 14:35:54.586499 14867 leader_election.cc:288] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [CANDIDATE]: Term 3 election: Requested vote from peers 4d882bd3213f40e182237f47cbaefda5 (127.2.103.126:34059), 2abc19ebb22040029f5122671c37a3d9 (127.2.103.125:40859)
I20260811 14:35:54.586696 14869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.586585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.0.0.1:41112)
0811 14:35:54.586639 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.586668 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 14:35:54.586732 13657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2abc19ebb22040029f5122671c37a3d9"
I20260811 14:35:54.586844 13657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 2 FOLLOWER]: Advancing to term 3
I20260811 14:35:54.587109 14869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.586971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.0.0.1:41114)
0811 14:35:54.587027 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.587058 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 14:35:54.587194 13931 leader_election.cc:334] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 4d882bd3213f40e182237f47cbaefda5 (127.2.103.126:34059): Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
I20260811 14:35:54.587646 13657 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7e1ff4603b74cfa81ef09f40f4dc030 in term 3.
I20260811 14:35:54.587853 13929 leader_election.cc:302] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2abc19ebb22040029f5122671c37a3d9, c7e1ff4603b74cfa81ef09f40f4dc030; no voters: 4d882bd3213f40e182237f47cbaefda5
I20260811 14:35:54.587968 14867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 14:35:54.588083 14867 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [term 3 LEADER]: Becoming Leader. State: Replica: c7e1ff4603b74cfa81ef09f40f4dc030, State: Running, Role: LEADER
I20260811 14:35:54.588199 14867 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false } }
I20260811 14:35:54.588598 14871 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7e1ff4603b74cfa81ef09f40f4dc030. Latest consensus state: current_term: 3 leader_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false } } }
I20260811 14:35:54.588703 14871 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:54.588925 14876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:54.589457 14876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:54.589550 14876 catalog_manager.cc:1259] Loaded cluster ID: 2af9f7732d9744fb80bf2acaaec01923
I20260811 14:35:54.589587 14876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:35:54.589771 14876 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:54.589857 14876 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: Loaded TSK: 0
I20260811 14:35:54.590008 14876 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:35:54.660194 13957 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57896:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:35:54.662024 13957 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:35:54.663324 14869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.663202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.0.0.1:41134)
0811 14:35:54.663259 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.663295 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 14:35:54.663353 13657 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [term 3 FOLLOWER]: Refusing update from remote peer c7e1ff4603b74cfa81ef09f40f4dc030: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 14:35:54.663647 14871 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 14:35:54.663785 14869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.663673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.0.0.1:41138)
0811 14:35:54.663730 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.663762 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 14:35:54.663923 13931 consensus_peers.cc:487] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 -> Peer 4d882bd3213f40e182237f47cbaefda5 (127.2.103.126:34059): Couldn't send request to peer 4d882bd3213f40e182237f47cbaefda5. Status: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 14:35:54.664364 14879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7e1ff4603b74cfa81ef09f40f4dc030. Latest consensus state: current_term: 3 leader_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false } } }
I20260811 14:35:54.664475 14879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:35:54.664933 14867 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false } } }
I20260811 14:35:54.665032 14867 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:54.665130 14879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4d882bd3213f40e182237f47cbaefda5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 34059 } } peers { permanent_uuid: "2abc19ebb22040029f5122671c37a3d9" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 40859 } attrs { promote: false } } peers { permanent_uuid: "c7e1ff4603b74cfa81ef09f40f4dc030" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33113 } attrs { promote: false } } }
I20260811 14:35:54.665278 14879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2abc19ebb22040029f5122671c37a3d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:35:54.670222 14898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.667373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42308 (local address 127.2.103.65:45703)
0811 14:35:54.667632 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:54.667634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:54.667641 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:35:54.667664 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:54.667665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:54.667683 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:54.667766 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:54.667861 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.668335 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.669431 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.669877 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.669952 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:54.669965 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:54.670106 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 14:35:54.670150 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":64,"threads_started":1}
I20260811 14:35:54.670243 14896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.667207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:45703 (local address 127.0.0.1:42308)
0811 14:35:54.667493 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.667502 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:54.667545 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:54.667768 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:54.667769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:54.667770 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:54.667807 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:54.667809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.668418 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.668420 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:54.669322 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:54.669324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.669963 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.669964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:54.670026 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:54.670031 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:35:54.670087 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 14:35:54.670128 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":65,"thread_start_us":65,"threads_started":1}
I20260811 14:35:54.670763 14869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.666915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:44753 (local address 127.0.0.1:48352)
0811 14:35:54.666968 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.666979 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:54.667028 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:54.667433 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:54.667434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:54.667437 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:54.667493 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:54.667495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.668497 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.668498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:54.669712 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:54.669715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.670481 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.670482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:54.670544 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:54.670548 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:35:54.670600 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 14:35:54.670676 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":48}
I20260811 14:35:54.670771 14155 tablet_service.cc:1467] Processing CreateTablet for tablet 1eb58b6aa4b648d0963d4d07dc0a3c9b (DEFAULT_TABLE table=test-table [id=89df7b1b062e4d6198873c2b484d50d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:54.670938 14894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.667025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:41451 (local address 127.0.0.1:37088)
0811 14:35:54.667155 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.667163 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:54.667202 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:54.667487 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:54.667487 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:54.667489 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:54.667546 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:54.667548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.668488 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.668489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:54.669713 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:54.669715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.670537 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.670538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:54.670704 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:54.670708 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:35:54.670754 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 14:35:54.670794 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":53,"thread_start_us":92,"threads_started":1}
I20260811 14:35:54.670951 14155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eb58b6aa4b648d0963d4d07dc0a3c9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:54.671159 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.667097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37088 (local address 127.2.103.67:41451)
0811 14:35:54.667285 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:54.667287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:54.667295 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:35:54.667320 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:54.667321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:54.667333 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:54.667386 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:54.667624 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.668392 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.669806 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.670400 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.670494 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:54.670508 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:54.671039 (+ 531us) server_negotiation.cc:299] Negotiation successful
0811 14:35:54.671092 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":85,"threads_started":1}
I20260811 14:35:54.671379 14596 tablet_service.cc:1467] Processing CreateTablet for tablet 1eb58b6aa4b648d0963d4d07dc0a3c9b (DEFAULT_TABLE table=test-table [id=89df7b1b062e4d6198873c2b484d50d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:54.671540 14596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eb58b6aa4b648d0963d4d07dc0a3c9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:54.672343 14899 tablet_bootstrap.cc:492] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P d6025d4830f2498cb4f8430ac71bc2d0: Bootstrap starting.
I20260811 14:35:54.672434 14888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.667004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48352 (local address 127.2.103.66:44753)
0811 14:35:54.667244 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:54.667247 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:54.667258 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:35:54.667289 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:54.667290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:54.667304 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:54.667561 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:54.667629 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.668391 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.669818 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.670383 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.670455 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:54.670467 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:54.670655 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 14:35:54.670697 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":59,"threads_started":1}
I20260811 14:35:54.672822 14899 tablet_bootstrap.cc:654] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P d6025d4830f2498cb4f8430ac71bc2d0: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:54.672894 14900 tablet_bootstrap.cc:492] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 570caddfa3a84f0682765c59723893dd: Bootstrap starting.
I20260811 14:35:54.672994 14361 tablet_service.cc:1467] Processing CreateTablet for tablet 1eb58b6aa4b648d0963d4d07dc0a3c9b (DEFAULT_TABLE table=test-table [id=89df7b1b062e4d6198873c2b484d50d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:54.673130 14361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eb58b6aa4b648d0963d4d07dc0a3c9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:54.673285 14900 tablet_bootstrap.cc:654] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 570caddfa3a84f0682765c59723893dd: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:54.673610 14899 tablet_bootstrap.cc:492] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P d6025d4830f2498cb4f8430ac71bc2d0: No bootstrap required, opened a new log
I20260811 14:35:54.673673 14899 ts_tablet_manager.cc:1397] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P d6025d4830f2498cb4f8430ac71bc2d0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:54.673808 14899 raft_consensus.cc:348] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P d6025d4830f2498cb4f8430ac71bc2d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 } } peers { permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 } } peers { permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 } }
I20260811 14:35:54.673895 14899 raft_consensus.cc:374] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P d6025d4830f2498cb4f8430ac71bc2d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:54.673884 14900 tablet_bootstrap.cc:492] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 570caddfa3a84f0682765c59723893dd: No bootstrap required, opened a new log
I20260811 14:35:54.673921 14899 raft_consensus.cc:729] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P d6025d4830f2498cb4f8430ac71bc2d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6025d4830f2498cb4f8430ac71bc2d0, State: Initialized, Role: FOLLOWER
I20260811 14:35:54.673950 14900 ts_tablet_manager.cc:1397] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 570caddfa3a84f0682765c59723893dd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:54.673988 14899 consensus_queue.cc:260] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P d6025d4830f2498cb4f8430ac71bc2d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 } } peers { permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 } } peers { permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 } }
I20260811 14:35:54.674083 14900 raft_consensus.cc:348] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 570caddfa3a84f0682765c59723893dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 } } peers { permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 } } peers { permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 } }
I20260811 14:35:54.674157 14899 ts_tablet_manager.cc:1428] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P d6025d4830f2498cb4f8430ac71bc2d0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:54.674170 14900 raft_consensus.cc:374] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 570caddfa3a84f0682765c59723893dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:54.674199 14900 raft_consensus.cc:729] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 570caddfa3a84f0682765c59723893dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 570caddfa3a84f0682765c59723893dd, State: Initialized, Role: FOLLOWER
I20260811 14:35:54.674249 14900 consensus_queue.cc:260] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 570caddfa3a84f0682765c59723893dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 } } peers { permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 } } peers { permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 } }
I20260811 14:35:54.674540 14859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.674443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.65:60151)
0811 14:35:54.674485 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.674514 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260811 14:35:54.674638 14908 tablet_bootstrap.cc:492] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6: Bootstrap starting.
I20260811 14:35:54.674965 14859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.674869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.65:52113)
0811 14:35:54.674905 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.674939 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260811 14:35:54.675105 14900 ts_tablet_manager.cc:1428] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 570caddfa3a84f0682765c59723893dd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:54.675472 14908 tablet_bootstrap.cc:654] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:54.675673 14862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.675583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.67:37847)
0811 14:35:54.675624 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.675647 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260811 14:35:54.675788 14705 heartbeater.cc:503] Master 127.2.103.124:33113 was elected leader, sending a full tablet report...
I20260811 14:35:54.676049 14862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.675969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.67:49339)
0811 14:35:54.676005 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.676027 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260811 14:35:54.676158 14908 tablet_bootstrap.cc:492] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6: No bootstrap required, opened a new log
I20260811 14:35:54.676216 14908 ts_tablet_manager.cc:1397] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:35:54.676339 14908 raft_consensus.cc:348] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 } } peers { permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 } } peers { permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 } }
I20260811 14:35:54.676414 14908 raft_consensus.cc:374] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:54.676437 14908 raft_consensus.cc:729] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d80c3098fb34a9b97eda8e7427eb3d6, State: Initialized, Role: FOLLOWER
I20260811 14:35:54.676489 14908 consensus_queue.cc:260] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 } } peers { permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 } } peers { permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 } }
I20260811 14:35:54.676689 14908 ts_tablet_manager.cc:1428] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:54.676957 14865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.676873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.66:49003)
0811 14:35:54.676904 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.676933 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260811 14:35:54.677281 14472 heartbeater.cc:503] Master 127.2.103.124:33113 was elected leader, sending a full tablet report...
I20260811 14:35:54.677337 14865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.677206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.66:35261)
0811 14:35:54.677275 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.677300 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 14:35:54.713867 14258 tablet.cc:2369] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P d6025d4830f2498cb4f8430ac71bc2d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:35:54.743228 14924 raft_consensus.cc:482] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:35:54.743374 14924 raft_consensus.cc:504] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 } } peers { permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 } } peers { permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 } }
I20260811 14:35:54.743855 14924 leader_election.cc:288] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 570caddfa3a84f0682765c59723893dd (127.2.103.67:41451), d6025d4830f2498cb4f8430ac71bc2d0 (127.2.103.65:45703)
I20260811 14:35:54.747071 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.744123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:47943 (local address 127.2.103.67:41451)
0811 14:35:54.744185 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:54.744188 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:54.744201 (+ 13us) server_negotiation.cc:407] Connection header received
0811 14:35:54.744241 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:54.744243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:54.744257 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:54.744337 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:54.744590 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.745149 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.746335 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.746701 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.746760 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:54.746768 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:54.746950 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 14:35:54.747000 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 14:35:54.747061 14865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.744038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:41451 (local address 127.2.103.66:47943)
0811 14:35:54.744107 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.744120 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:54.744179 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:54.744342 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:54.744344 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:54.744347 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:54.744470 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:54.744473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.745242 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.745244 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:54.746229 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:54.746234 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.746791 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.746792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:54.746869 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:54.746873 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:35:54.746931 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 14:35:54.746964 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":36}
I20260811 14:35:54.747176 14898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.744255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:32835 (local address 127.2.103.65:45703)
0811 14:35:54.744312 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:54.744314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:54.744460 (+ 146us) server_negotiation.cc:407] Connection header received
0811 14:35:54.744504 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:54.744505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:54.744518 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:54.744569 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:54.744735 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.745209 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.746259 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.746760 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.746837 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:54.746849 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:54.747052 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 14:35:54.747101 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 14:35:54.747222 14925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.744175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:45703 (local address 127.2.103.66:32835)
0811 14:35:54.744376 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.744384 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:54.744447 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:54.744591 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:54.744592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:54.744594 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:54.744664 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:54.744666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.745326 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.745327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:54.746154 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:54.746157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.746873 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.746873 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:54.746963 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:54.746968 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:35:54.747025 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 14:35:54.747064 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":43,"thread_start_us":129,"threads_started":1}
I20260811 14:35:54.747447 14633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eb58b6aa4b648d0963d4d07dc0a3c9b" candidate_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "570caddfa3a84f0682765c59723893dd" is_pre_election: true
I20260811 14:35:54.747539 14633 raft_consensus.cc:2454] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 570caddfa3a84f0682765c59723893dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d80c3098fb34a9b97eda8e7427eb3d6 in term 0.
I20260811 14:35:54.747454 14189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eb58b6aa4b648d0963d4d07dc0a3c9b" candidate_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" is_pre_election: true
I20260811 14:35:54.747609 14189 raft_consensus.cc:2454] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P d6025d4830f2498cb4f8430ac71bc2d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d80c3098fb34a9b97eda8e7427eb3d6 in term 0.
I20260811 14:35:54.747732 14295 leader_election.cc:302] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d80c3098fb34a9b97eda8e7427eb3d6, 570caddfa3a84f0682765c59723893dd; no voters:
I20260811 14:35:54.747905 14924 raft_consensus.cc:2781] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:35:54.747959 14924 raft_consensus.cc:482] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:35:54.747982 14924 raft_consensus.cc:3037] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:54.748739 14924 raft_consensus.cc:504] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 } } peers { permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 } } peers { permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 } }
I20260811 14:35:54.748890 14924 leader_election.cc:288] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [CANDIDATE]: Term 1 election: Requested vote from peers 570caddfa3a84f0682765c59723893dd (127.2.103.67:41451), d6025d4830f2498cb4f8430ac71bc2d0 (127.2.103.65:45703)
I20260811 14:35:54.749066 14633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eb58b6aa4b648d0963d4d07dc0a3c9b" candidate_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "570caddfa3a84f0682765c59723893dd"
I20260811 14:35:54.749082 14189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eb58b6aa4b648d0963d4d07dc0a3c9b" candidate_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6025d4830f2498cb4f8430ac71bc2d0"
I20260811 14:35:54.749153 14189 raft_consensus.cc:3037] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P d6025d4830f2498cb4f8430ac71bc2d0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:54.749157 14633 raft_consensus.cc:3037] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 570caddfa3a84f0682765c59723893dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:54.749775 14189 raft_consensus.cc:2454] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P d6025d4830f2498cb4f8430ac71bc2d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d80c3098fb34a9b97eda8e7427eb3d6 in term 1.
I20260811 14:35:54.749835 14633 raft_consensus.cc:2454] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 570caddfa3a84f0682765c59723893dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d80c3098fb34a9b97eda8e7427eb3d6 in term 1.
I20260811 14:35:54.749958 14295 leader_election.cc:302] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d80c3098fb34a9b97eda8e7427eb3d6, 570caddfa3a84f0682765c59723893dd; no voters:
I20260811 14:35:54.750088 14924 raft_consensus.cc:2781] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:54.750268 14924 raft_consensus.cc:686] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 1 LEADER]: Becoming Leader. State: Replica: 3d80c3098fb34a9b97eda8e7427eb3d6, State: Running, Role: LEADER
I20260811 14:35:54.750353 14924 consensus_queue.cc:237] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 } } peers { permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 } } peers { permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 } }
I20260811 14:35:54.751060 14925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.750965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.66:42509)
0811 14:35:54.751003 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.751033 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260811 14:35:54.751186 13957 catalog_manager.cc:5495] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d80c3098fb34a9b97eda8e7427eb3d6 (127.2.103.66). New cstate: current_term: 1 leader_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d80c3098fb34a9b97eda8e7427eb3d6" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44753 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 } health_report { overall_health: UNKNOWN } } }
I20260811 14:35:54.766885 14888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.764430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48364 (local address 127.2.103.66:44753)
0811 14:35:54.764491 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:54.764493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:54.764501 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:35:54.764553 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:54.764554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:54.764574 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:54.764654 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:54.764791 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.765270 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.766022 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:54.766224 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:54.766300 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:54.766470 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:54.766579 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:54.766731 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:54.766770 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 14:35:54.766795 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 14:35:54.768183 14633 raft_consensus.cc:1261] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 570caddfa3a84f0682765c59723893dd [term 1 FOLLOWER]: Refusing update from remote peer 3d80c3098fb34a9b97eda8e7427eb3d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:54.768302 14189 raft_consensus.cc:1261] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P d6025d4830f2498cb4f8430ac71bc2d0 [term 1 FOLLOWER]: Refusing update from remote peer 3d80c3098fb34a9b97eda8e7427eb3d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:54.768532 14928 consensus_queue.cc:1035] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "570caddfa3a84f0682765c59723893dd" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 41451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:54.768671 14924 consensus_queue.cc:1035] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6025d4830f2498cb4f8430ac71bc2d0" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 45703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:54.769302 14254 heartbeater.cc:503] Master 127.2.103.124:33113 was elected leader, sending a full tablet report...
I20260811 14:35:54.769578 14862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.769414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.67:44747)
0811 14:35:54.769508 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.769546 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 14:35:54.769732 14859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.769625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.65:50993)
0811 14:35:54.769670 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.769700 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 14:35:54.771219 14938 mvcc.cc:205] Tried to move back new op lower bound from 7317335878727847936 to 7317335878658109440. Current Snapshot: MvccSnapshot[applied={T|T < 7317335878727847936}]
I20260811 14:35:54.771517 14862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.771411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.67:33183)
0811 14:35:54.771455 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.771483 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 14:35:54.771653 14859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.771564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.65:39811)
0811 14:35:54.771605 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.771629 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260811 14:35:54.771960 14925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:54.771807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:34059 (local address 127.2.103.66:59477)
0811 14:35:54.771888 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.771932 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:34059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 14:35:54.773939 13957 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:54.774032 13957 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:54.775343 13957 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7e1ff4603b74cfa81ef09f40f4dc030: Sending DeleteTablet for 3 replicas of tablet 1eb58b6aa4b648d0963d4d07dc0a3c9b
I20260811 14:35:54.775715 14361 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eb58b6aa4b648d0963d4d07dc0a3c9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:54 UTC) from {username='oryx', principal='oryx/127.2.103.124@KRBTEST.COM'} at 127.0.0.1:48352
I20260811 14:35:54.775743 14596 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eb58b6aa4b648d0963d4d07dc0a3c9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:54 UTC) from {username='oryx', principal='oryx/127.2.103.124@KRBTEST.COM'} at 127.0.0.1:37088
I20260811 14:35:54.775799 14155 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eb58b6aa4b648d0963d4d07dc0a3c9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:54 UTC) from {username='oryx', principal='oryx/127.2.103.124@KRBTEST.COM'} at 127.0.0.1:42308
I20260811 14:35:54.775987 14958 tablet_replica.cc:330] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P d6025d4830f2498cb4f8430ac71bc2d0: stopping tablet replica
I20260811 14:35:54.776048 14953 tablet_replica.cc:330] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6: stopping tablet replica
I20260811 14:35:54.776110 14953 raft_consensus.cc:2229] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:35:54.776209 14953 raft_consensus.cc:2258] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:54.776250 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 13893
I20260811 14:35:54.776327 14954 tablet_replica.cc:330] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 570caddfa3a84f0682765c59723893dd: stopping tablet replica
I20260811 14:35:54.776382 14954 raft_consensus.cc:2229] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 570caddfa3a84f0682765c59723893dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:54.776437 14954 raft_consensus.cc:2258] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 570caddfa3a84f0682765c59723893dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:54.785877 14953 ts_tablet_manager.cc:1905] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:54.787626 14954 ts_tablet_manager.cc:1905] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 570caddfa3a84f0682765c59723893dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:54.787652 14953 ts_tablet_manager.cc:1918] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:35:54.787708 14953 log.cc:1196] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/1eb58b6aa4b648d0963d4d07dc0a3c9b
I20260811 14:35:54.787992 14953 ts_tablet_manager.cc:1939] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 3d80c3098fb34a9b97eda8e7427eb3d6: Deleting consensus metadata
I20260811 14:35:54.789063 14954 ts_tablet_manager.cc:1918] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 570caddfa3a84f0682765c59723893dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:35:54.789108 14954 log.cc:1196] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 570caddfa3a84f0682765c59723893dd: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/1eb58b6aa4b648d0963d4d07dc0a3c9b
I20260811 14:35:54.789306 14954 ts_tablet_manager.cc:1939] T 1eb58b6aa4b648d0963d4d07dc0a3c9b P 570caddfa3a84f0682765c59723893dd: Deleting consensus metadata
I20260811 14:35:54.789620 13929 catalog_manager.cc:4841] TS 570caddfa3a84f0682765c59723893dd (127.2.103.67:41451): tablet 1eb58b6aa4b648d0963d4d07dc0a3c9b (table test-table [id=89df7b1b062e4d6198873c2b484d50d8]) successfully deleted
I20260811 14:35:54.790797 13929 catalog_manager.cc:4841] TS 3d80c3098fb34a9b97eda8e7427eb3d6 (127.2.103.66:44753): tablet 1eb58b6aa4b648d0963d4d07dc0a3c9b (table test-table [id=89df7b1b062e4d6198873c2b484d50d8]) successfully deleted
W20260811 14:35:54.793884 14894 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:54.793744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:45703 (local address 127.0.0.1:42318)
0811 14:35:54.793802 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:54.793836 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.65:45703: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 14:35:54.794044 13929 catalog_manager.cc:4584] TS d6025d4830f2498cb4f8430ac71bc2d0 (127.2.103.65:45703): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1eb58b6aa4b648d0963d4d07dc0a3c9b: Network error: Client connection negotiation failed: client connection to 127.2.103.65:45703: connect: Connection reset by peer (error 104)
I20260811 14:35:54.794152 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 14265
I20260811 14:35:54.803283 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 14484
I20260811 14:35:54.820293 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 13411
I20260811 14:35:54.834828 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 13729
2026-08-11T14:35:54Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6691 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:54 dist-test-slave-twwt krb5kdc[14961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:54 dist-test-slave-twwt krb5kdc[14961](info): set up 2 sockets
Aug 11 14:35:54 dist-test-slave-twwt krb5kdc[14961](info): commencing operation
krb5kdc: starting...
W20260811 14:35:56.126124 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:35:56 dist-test-slave-twwt krb5kdc[14961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:56Z Disabled control of system clock
WARNING: no policy specified for oryx/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.126.keytab.
Entry for principal oryx/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.126.keytab.
I20260811 14:35:56.254472 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:42013
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.126.keytab
--principal=oryx/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:39717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:42013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:56.342056 14989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:56.342350 14989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:56.342392 14989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:56.343658 14989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:56.343729 14989 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 14:35:56.343752 14989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:56.343786 14989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:56.343814 14989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:56.343840 14989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:56.345422 14989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:39717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:42013
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.126.keytab
--principal=oryx/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:42013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:56.345719 14989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:56.345976 14989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:56.351107 14995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:56.351094 14997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:56.351081 14999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:56.351200 14989 server_base.cc:1034] running on GCE node
I20260811 14:35:56.351544 14989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:56.351867 14989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:56.353016 14989 hybrid_clock.cc:648] HybridClock initialized: now 1786458956353003 us; error 26 us; skew 500 ppm
Aug 11 14:35:56 dist-test-slave-twwt krb5kdc[14961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458956, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:56.361590 14989 init.cc:377] Logged in from keytab as oryx/127.2.103.126@KRBTEST.COM (short username oryx)
I20260811 14:35:56.362085 14989 webserver.cc:460] Webserver started at http://127.2.103.126:35359/ using document root <none> and password file <none>
I20260811 14:35:56.362342 14989 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:56.362401 14989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:56.362529 14989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:56.363826 14989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "8cbe4bde4366476ba2fc7f4cd3824c25"
format_stamp: "Formatted at 2026-08-11 14:35:56 on dist-test-slave-twwt"
server_key: "ca04d274e0d8e3b8b1f8248d86395c60"
server_key_iv: "f4369c77a5d88e4ca3207e4bec92a4b9"
server_key_version: "encryptionkey@0"
I20260811 14:35:56.364176 14989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "8cbe4bde4366476ba2fc7f4cd3824c25"
format_stamp: "Formatted at 2026-08-11 14:35:56 on dist-test-slave-twwt"
server_key: "ca04d274e0d8e3b8b1f8248d86395c60"
server_key_iv: "f4369c77a5d88e4ca3207e4bec92a4b9"
server_key_version: "encryptionkey@0"
I20260811 14:35:56.365694 14989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:56.366555 15009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:56.366791 14989 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:56.366894 14989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "8cbe4bde4366476ba2fc7f4cd3824c25"
format_stamp: "Formatted at 2026-08-11 14:35:56 on dist-test-slave-twwt"
server_key: "ca04d274e0d8e3b8b1f8248d86395c60"
server_key_iv: "f4369c77a5d88e4ca3207e4bec92a4b9"
server_key_version: "encryptionkey@0"
I20260811 14:35:56.366961 14989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:56.377671 14989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:56.380142 14989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:56.380264 14989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:56.385229 14989 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:42013
I20260811 14:35:56.385332 15109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:42013 every 8 connection(s)
I20260811 14:35:56.385685 14989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:56.386467 15110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:56.388993 15110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25: Bootstrap starting.
I20260811 14:35:56.389684 15110 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:56.390010 15110 log.cc:824] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:56.390811 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 14989
I20260811 14:35:56.390894 15110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25: No bootstrap required, opened a new log
I20260811 14:35:56.390902 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:35:56.391129 2461 external_mini_cluster.cc:1428] Setting key e02ef85ecaf2c9929bd20ea7ac13764a
I20260811 14:35:56.393365 15110 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cbe4bde4366476ba2fc7f4cd3824c25" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42013 } }
I20260811 14:35:56.393546 15110 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:56.393612 15110 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cbe4bde4366476ba2fc7f4cd3824c25, State: Initialized, Role: FOLLOWER
I20260811 14:35:56.393806 15110 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cbe4bde4366476ba2fc7f4cd3824c25" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42013 } }
I20260811 14:35:56.393922 15110 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:56.393971 15110 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:56.394021 15110 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:56.394687 15110 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cbe4bde4366476ba2fc7f4cd3824c25" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42013 } }
I20260811 14:35:56.394819 15110 leader_election.cc:302] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cbe4bde4366476ba2fc7f4cd3824c25; no voters:
I20260811 14:35:56.395053 15110 leader_election.cc:288] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:56.395093 15121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:56.395215 15121 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25 [term 1 LEADER]: Becoming Leader. State: Replica: 8cbe4bde4366476ba2fc7f4cd3824c25, State: Running, Role: LEADER
I20260811 14:35:56.395380 15110 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:56.395412 15121 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cbe4bde4366476ba2fc7f4cd3824c25" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42013 } }
I20260811 14:35:56.395861 15122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cbe4bde4366476ba2fc7f4cd3824c25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cbe4bde4366476ba2fc7f4cd3824c25" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42013 } } }
I20260811 14:35:56.395972 15122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:56.396281 15128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cbe4bde4366476ba2fc7f4cd3824c25. Latest consensus state: current_term: 1 leader_uuid: "8cbe4bde4366476ba2fc7f4cd3824c25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cbe4bde4366476ba2fc7f4cd3824c25" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42013 } } }
I20260811 14:35:56.396441 15128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:56.396632 15136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:56.397239 15136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:56.399597 15136 catalog_manager.cc:1347] Generated new cluster ID: c19217f34d414d7981447040424d941d
I20260811 14:35:56.399665 15136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:35:56 dist-test-slave-twwt krb5kdc[14961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM
I20260811 14:35:56.405525 15120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:56.391722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34192 (local address 127.2.103.126:42013)
0811 14:35:56.391930 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:56.391939 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:56.391966 (+ 27us) server_negotiation.cc:407] Connection header received
0811 14:35:56.393353 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:56.393363 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:56.393428 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:56.393592 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:56.394364 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.395641 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.396795 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.397434 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.397540 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.400679 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:56.400683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:56.400684 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:56.400698 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:56.403541 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:56.403987 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:56.403991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:56.403991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:56.404133 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:56.404351 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:56.404351 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:56.404352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:56.404717 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:56.404907 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:56.404987 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 14:35:56.405276 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
I20260811 14:35:56.409569 15136 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:56.410310 15136 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:56.418699 15136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25: Generated new TSK 0
I20260811 14:35:56.418937 15136 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.65.keytab.
Entry for principal oryx/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.65.keytab.
I20260811 14:35:56.453344 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:42013
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.65.keytab
--principal=oryx/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:39717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:56.539055 15152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:56.539287 15152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:56.539328 15152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:56.540489 15152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:56.540562 15152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:56.540597 15152 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 14:35:56.540634 15152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:35:56.542239 15152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:39717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.65.keytab
--principal=oryx/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.103.126:42013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:56.542523 15152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:56.542811 15152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:56.548267 15158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:56.548256 15161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:56.548293 15152 server_base.cc:1034] running on GCE node
W20260811 14:35:56.548257 15164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:56.549024 15152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:56.549353 15152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:56.550535 15152 hybrid_clock.cc:648] HybridClock initialized: now 1786458956550525 us; error 56 us; skew 500 ppm
Aug 11 14:35:56 dist-test-slave-twwt krb5kdc[14961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458956, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:56.553334 15152 init.cc:377] Logged in from keytab as oryx/127.2.103.65@KRBTEST.COM (short username oryx)
I20260811 14:35:56.553824 15152 webserver.cc:460] Webserver started at http://127.2.103.65:34287/ using document root <none> and password file <none>
I20260811 14:35:56.554049 15152 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:56.554092 15152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:56.554191 15152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:56.555357 15152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "95fe30de2e3341b19420c5e84a242b83"
format_stamp: "Formatted at 2026-08-11 14:35:56 on dist-test-slave-twwt"
server_key: "3a165d9bb518b9c66d6503c29dfbeb5f"
server_key_iv: "ed93194af25e24f80b447093a83cbd0e"
server_key_version: "encryptionkey@0"
I20260811 14:35:56.555687 15152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "95fe30de2e3341b19420c5e84a242b83"
format_stamp: "Formatted at 2026-08-11 14:35:56 on dist-test-slave-twwt"
server_key: "3a165d9bb518b9c66d6503c29dfbeb5f"
server_key_iv: "ed93194af25e24f80b447093a83cbd0e"
server_key_version: "encryptionkey@0"
I20260811 14:35:56.556991 15152 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:35:56.557668 15174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:56.557857 15152 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:56.557952 15152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "95fe30de2e3341b19420c5e84a242b83"
format_stamp: "Formatted at 2026-08-11 14:35:56 on dist-test-slave-twwt"
server_key: "3a165d9bb518b9c66d6503c29dfbeb5f"
server_key_iv: "ed93194af25e24f80b447093a83cbd0e"
server_key_version: "encryptionkey@0"
I20260811 14:35:56.558012 15152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:56.576597 15152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:56.579111 15152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:56.579264 15152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:56.579857 15152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:56.579905 15152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:56.579938 15152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:56.579984 15152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:56.587258 15152 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:35599
I20260811 14:35:56.587316 15377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:35599 every 8 connection(s)
I20260811 14:35:56.587646 15152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:35:56 dist-test-slave-twwt krb5kdc[14961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458956, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM
I20260811 14:35:56.589525 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 15152
I20260811 14:35:56.589660 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:35:56.589917 2461 external_mini_cluster.cc:1428] Setting key 103c77b19f3293ec474f29e8b7d1c175
I20260811 14:35:56.591578 15120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:56.580399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:46733 (local address 127.2.103.126:42013)
0811 14:35:56.580480 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:56.580483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:56.580682 (+ 199us) server_negotiation.cc:407] Connection header received
0811 14:35:56.581206 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:56.581209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:56.581227 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:56.581308 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:56.582274 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.583172 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.584940 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.585265 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.585329 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.588297 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:56.588299 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:56.588300 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:56.588308 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:56.590033 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:56.590484 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:56.590486 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:56.590486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:56.590555 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:56.590876 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:56.590877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:56.590878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:56.591002 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:56.591083 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:56.591388 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 14:35:56.591485 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 14:35:56.591753 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:56.580360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42013 (local address 127.2.103.65:46733)
0811 14:35:56.580541 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:56.580586 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:56.581106 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:56.581417 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:56.581421 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:56.581827 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:56.582151 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:56.582157 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.583306 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.583307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:56.584837 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:56.584840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.585357 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.585358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:56.585540 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.586214 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:56.586228 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:56.588189 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:56.590139 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:56.590148 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:56.590151 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:56.590409 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:56.590657 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:56.590658 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:56.590658 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:56.590785 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:56.591074 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:56.591082 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:56.591226 (+ 144us) client_negotiation.cc:769] Sending connection context
0811 14:35:56.591336 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 14:35:56.591510 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":107,"threads_started":1}
I20260811 14:35:56.592877 15379 heartbeater.cc:347] Connected to a master server at 127.2.103.126:42013
I20260811 14:35:56.592973 15379 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:56.593228 15379 heartbeater.cc:511] Master 127.2.103.126:42013 requested a full tablet report, sending...
I20260811 14:35:56.593843 15034 ts_manager.cc:194] Registered new tserver with Master: 95fe30de2e3341b19420c5e84a242b83 (127.2.103.65:35599)
I20260811 14:35:56.594502 15034 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:46733
WARNING: no policy specified for oryx/127.2.103.66@KRBTEST.COM; defaulting to no policy
I20260811 14:35:56.600904 15120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:56.594984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:34109 (local address 127.2.103.126:42013)
0811 14:35:56.595026 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:56.595029 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:56.595168 (+ 139us) server_negotiation.cc:407] Connection header received
0811 14:35:56.595217 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:56.595218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:56.595230 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:56.595297 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:56.595661 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.596149 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.596942 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.597144 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.597226 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.598211 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:56.598213 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:56.598214 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:56.598222 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:56.599672 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:56.600070 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:56.600071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:56.600072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:56.600135 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:56.600361 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:56.600362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:56.600363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:56.600478 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:56.600552 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:56.600723 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 14:35:56.600805 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 14:35:56.600904 15389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:56.594920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42013 (local address 127.2.103.65:34109)
0811 14:35:56.595088 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:56.595099 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:56.595163 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:56.595299 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:56.595300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:56.595524 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:56.595587 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:56.595589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.596252 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.596254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:56.596854 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:56.596857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.597224 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.597225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:56.597290 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.597790 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:56.597795 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:56.598113 (+ 318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:56.599784 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:56.599786 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:56.599786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:56.599992 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:56.600213 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:56.600214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:56.600215 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:56.600289 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:56.600555 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:56.600557 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:56.600604 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 14:35:56.600684 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 14:35:56.600801 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":83,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.103.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.66.keytab.
Entry for principal oryx/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.66.keytab.
I20260811 14:35:56.617661 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:42013
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.66.keytab
--principal=oryx/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:39717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:56.703855 15394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:56.704075 15394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:56.704104 15394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:56.705204 15394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:56.705240 15394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:56.705260 15394 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 14:35:56.705278 15394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:35:56.706786 15394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:39717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.66.keytab
--principal=oryx/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.103.126:42013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:56.706998 15394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:56.707198 15394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:56.712571 15402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:56.712616 15407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:56.712749 15403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:56.713042 15394 server_base.cc:1034] running on GCE node
I20260811 14:35:56.713222 15394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:56.713455 15394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:56.714594 15394 hybrid_clock.cc:648] HybridClock initialized: now 1786458956714576 us; error 36 us; skew 500 ppm
Aug 11 14:35:56 dist-test-slave-twwt krb5kdc[14961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458956, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:56.717298 15394 init.cc:377] Logged in from keytab as oryx/127.2.103.66@KRBTEST.COM (short username oryx)
I20260811 14:35:56.717806 15394 webserver.cc:460] Webserver started at http://127.2.103.66:35745/ using document root <none> and password file <none>
I20260811 14:35:56.718086 15394 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:56.718159 15394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:56.718312 15394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:56.719488 15394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "448de06301414a92bcc89b462b503faa"
format_stamp: "Formatted at 2026-08-11 14:35:56 on dist-test-slave-twwt"
server_key: "1ee2214a47e7b9cda4655642b05928b5"
server_key_iv: "9c8d11a138251a331210452b0dc63add"
server_key_version: "encryptionkey@0"
I20260811 14:35:56.719874 15394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "448de06301414a92bcc89b462b503faa"
format_stamp: "Formatted at 2026-08-11 14:35:56 on dist-test-slave-twwt"
server_key: "1ee2214a47e7b9cda4655642b05928b5"
server_key_iv: "9c8d11a138251a331210452b0dc63add"
server_key_version: "encryptionkey@0"
I20260811 14:35:56.721377 15394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:35:56.722343 15416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:56.722589 15394 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:35:56.722761 15394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "448de06301414a92bcc89b462b503faa"
format_stamp: "Formatted at 2026-08-11 14:35:56 on dist-test-slave-twwt"
server_key: "1ee2214a47e7b9cda4655642b05928b5"
server_key_iv: "9c8d11a138251a331210452b0dc63add"
server_key_version: "encryptionkey@0"
I20260811 14:35:56.722905 15394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:56.729965 15394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:56.732323 15394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:56.732501 15394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:56.733204 15394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:56.733263 15394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:56.733295 15394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:56.733309 15394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 14:35:56 dist-test-slave-twwt krb5kdc[14961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458956, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM
I20260811 14:35:56.741091 15394 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:46681
I20260811 14:35:56.741142 15615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:46681 every 8 connection(s)
I20260811 14:35:56.741549 15394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:35:56.743633 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 15394
I20260811 14:35:56.743779 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:35:56.743790 15120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:56.733697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:55119 (local address 127.2.103.126:42013)
0811 14:35:56.733774 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:56.733778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:56.733967 (+ 189us) server_negotiation.cc:407] Connection header received
0811 14:35:56.734474 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:56.734475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:56.734491 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:56.734568 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:56.735349 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.735863 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.736914 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.737247 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.737324 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.740344 (+ 3020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:56.740346 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:56.740347 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:56.740357 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:56.742183 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:56.742722 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:56.742723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:56.742723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:56.742784 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:56.743013 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:56.743014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:56.743014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:56.743130 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:56.743182 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:56.743565 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 14:35:56.743668 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 14:35:56.743891 15431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:56.733663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42013 (local address 127.2.103.66:55119)
0811 14:35:56.733829 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:56.733872 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:56.734379 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:56.734783 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:56.734788 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:56.735041 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:56.735233 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:56.735238 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.735977 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.735978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:56.736815 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:56.736818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.737334 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.737335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:56.737500 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.738006 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:56.738014 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:56.740180 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:56.742265 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:56.742270 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:56.742272 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:56.742584 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:56.742841 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:56.742842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:56.742842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:56.742949 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:56.743213 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:56.743227 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:56.743379 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 14:35:56.743478 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 14:35:56.743623 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":85,"threads_started":1}
I20260811 14:35:56.743990 2461 external_mini_cluster.cc:1428] Setting key 34c80b606dcd93e78e4f7c689a73029f
I20260811 14:35:56.744931 15616 heartbeater.cc:347] Connected to a master server at 127.2.103.126:42013
I20260811 14:35:56.745061 15616 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:56.745278 15616 heartbeater.cc:511] Master 127.2.103.126:42013 requested a full tablet report, sending...
I20260811 14:35:56.745682 15033 ts_manager.cc:194] Registered new tserver with Master: 448de06301414a92bcc89b462b503faa (127.2.103.66:46681)
I20260811 14:35:56.746110 15033 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:55119
I20260811 14:35:56.752667 15625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:56.746176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42013 (local address 127.2.103.66:38957)
0811 14:35:56.746315 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:56.746327 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:56.746390 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:56.746539 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:56.746540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:56.746828 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:56.746896 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:56.746898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.747620 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.747621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:56.748219 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:56.748221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.748667 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.748667 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:56.748718 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.749209 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:56.749222 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:56.749518 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:56.751552 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:56.751553 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:56.751553 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:56.751729 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:56.751965 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:56.751965 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:56.751966 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:56.752019 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:56.752401 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:56.752401 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:56.752446 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 14:35:56.752499 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 14:35:56.752584 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
I20260811 14:35:56.752880 15120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:56.746082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:38957 (local address 127.2.103.126:42013)
0811 14:35:56.746139 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:56.746141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:56.746395 (+ 254us) server_negotiation.cc:407] Connection header received
0811 14:35:56.746446 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:56.746447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:56.746459 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:56.746520 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:56.747010 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.747524 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.748292 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.748584 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.748662 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.749617 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:56.749618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:56.749619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:56.749627 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:56.751445 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:56.751806 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:56.751807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:56.751808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:56.751877 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:56.752089 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:56.752090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:56.752091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:56.752329 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:56.752409 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:56.752547 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 14:35:56.752646 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for oryx/127.2.103.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.103.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.67.keytab.
Entry for principal oryx/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.67.keytab.
I20260811 14:35:56.777503 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:42013
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.67.keytab
--principal=oryx/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:39717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:56.864641 15628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:56.864907 15628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:56.864955 15628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:56.866211 15628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:56.866282 15628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:56.866320 15628 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 14:35:56.866358 15628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:35:56.868196 15628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:39717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.67.keytab
--principal=oryx/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.103.126:42013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:56.868517 15628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:56.868767 15628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:56.874215 15642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:56.874213 15637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:56.874269 15640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:56.874508 15628 server_base.cc:1034] running on GCE node
I20260811 14:35:56.874776 15628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:56.875090 15628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:56.876269 15628 hybrid_clock.cc:648] HybridClock initialized: now 1786458956876253 us; error 27 us; skew 500 ppm
Aug 11 14:35:56 dist-test-slave-twwt krb5kdc[14961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458956, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:56.879271 15628 init.cc:377] Logged in from keytab as oryx/127.2.103.67@KRBTEST.COM (short username oryx)
I20260811 14:35:56.879706 15628 webserver.cc:460] Webserver started at http://127.2.103.67:44365/ using document root <none> and password file <none>
I20260811 14:35:56.879911 15628 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:56.879961 15628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:56.880053 15628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:56.881098 15628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "8ef83f76be0f497a81c571f8def5ecaa"
format_stamp: "Formatted at 2026-08-11 14:35:56 on dist-test-slave-twwt"
server_key: "18783b2a6072f6ce9cdb2c315987fd18"
server_key_iv: "63b70d17d7dc2aa714c237aecd372f3d"
server_key_version: "encryptionkey@0"
I20260811 14:35:56.881474 15628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "8ef83f76be0f497a81c571f8def5ecaa"
format_stamp: "Formatted at 2026-08-11 14:35:56 on dist-test-slave-twwt"
server_key: "18783b2a6072f6ce9cdb2c315987fd18"
server_key_iv: "63b70d17d7dc2aa714c237aecd372f3d"
server_key_version: "encryptionkey@0"
I20260811 14:35:56.882906 15628 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 14:35:56.883656 15651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:56.883908 15628 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:35:56.884003 15628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "8ef83f76be0f497a81c571f8def5ecaa"
format_stamp: "Formatted at 2026-08-11 14:35:56 on dist-test-slave-twwt"
server_key: "18783b2a6072f6ce9cdb2c315987fd18"
server_key_iv: "63b70d17d7dc2aa714c237aecd372f3d"
server_key_version: "encryptionkey@0"
I20260811 14:35:56.884059 15628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:56.898274 15628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:56.900779 15628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:56.900903 15628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:56.901389 15628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:35:56.901439 15628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:56.901468 15628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:35:56.901484 15628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 14:35:56 dist-test-slave-twwt krb5kdc[14961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458956, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM
I20260811 14:35:56.909021 15628 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:37791
I20260811 14:35:56.909090 15865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:37791 every 8 connection(s)
I20260811 14:35:56.909389 15628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:35:56.912379 15120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:56.902008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:48429 (local address 127.2.103.126:42013)
0811 14:35:56.902067 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:56.902068 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:56.902315 (+ 247us) server_negotiation.cc:407] Connection header received
0811 14:35:56.902984 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:56.902986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:56.903002 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:56.903054 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:56.903707 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.904525 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.906233 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.906559 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.906688 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.909560 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:56.909562 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:56.909562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:56.909571 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:56.910950 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:56.911373 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:56.911374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:56.911375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:56.911440 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:56.911694 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:56.911695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:56.911695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:56.911825 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:56.911898 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:56.912197 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 14:35:56.912283 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 14:35:56.912586 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:56.901998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42013 (local address 127.2.103.67:48429)
0811 14:35:56.902180 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:56.902220 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:56.902862 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:56.903131 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:56.903135 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:56.903376 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:56.903564 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:56.903568 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.904642 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.904643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:56.906104 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:56.906108 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.906719 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.906721 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:56.906882 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.907435 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:56.907447 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:56.909468 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:56.911034 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:56.911041 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:56.911044 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:56.911284 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:56.911513 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:56.911513 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:56.911514 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:56.911626 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:56.911901 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:56.911909 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:56.912050 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 14:35:56.912158 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 14:35:56.912341 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260811 14:35:56.913146 15867 heartbeater.cc:347] Connected to a master server at 127.2.103.126:42013
I20260811 14:35:56.913237 15867 heartbeater.cc:464] Registering TS with master...
I20260811 14:35:56.913460 15867 heartbeater.cc:511] Master 127.2.103.126:42013 requested a full tablet report, sending...
I20260811 14:35:56.913889 15034 ts_manager.cc:194] Registered new tserver with Master: 8ef83f76be0f497a81c571f8def5ecaa (127.2.103.67:37791)
I20260811 14:35:56.913964 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 15628
I20260811 14:35:56.914067 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:35:56.914291 2461 external_mini_cluster.cc:1428] Setting key 325211004a58dce4b6f1061b73add732
I20260811 14:35:56.914391 15034 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:48429
I20260811 14:35:56.915228 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:35:56.921434 15876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:56.914281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42013 (local address 127.2.103.67:36639)
0811 14:35:56.914432 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:56.914445 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:56.914498 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:56.914693 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:56.914694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:56.915519 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:56.915575 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:56.915577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.916454 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.916455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:56.916955 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:56.916957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.917443 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.917445 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:56.917521 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.917965 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:56.917971 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:56.918406 (+ 435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:56.920420 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:56.920421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:56.920421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:56.920591 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:56.920827 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:56.920828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:56.920828 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:56.920886 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:56.921180 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:56.921181 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:56.921220 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 14:35:56.921272 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 14:35:56.921363 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260811 14:35:56.921512 15120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:56.914355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:36639 (local address 127.2.103.126:42013)
0811 14:35:56.914528 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:56.914530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:56.914538 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:35:56.914558 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:56.914559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:56.914569 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:56.914646 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:56.915663 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.916348 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.917035 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.917346 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.917420 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.918497 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:56.918498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:56.918499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:56.918506 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:56.920332 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:56.920671 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:56.920673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:56.920673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:56.920749 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:56.920957 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:56.920957 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:56.920958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:56.921110 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:56.921194 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:56.921321 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 14:35:56.921409 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
Aug 11 14:35:56 dist-test-slave-twwt krb5kdc[14961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458956, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 14:35:56 dist-test-slave-twwt krb5kdc[14961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:35:56 dist-test-slave-twwt krb5kdc[14961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 14:35:56.940753 15120 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:56.935148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34208 (local address 127.2.103.126:42013)
0811 14:35:56.935216 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:56.935218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:56.935227 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:35:56.935255 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:56.935256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:56.935265 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:56.935326 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:56.935915 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.936427 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.937164 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.937346 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.937400 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.940685 (+ 3285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34208: BlockingRecv error: recv got EOF from 127.0.0.1:34208 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 11 14:35:56 dist-test-slave-twwt krb5kdc[14961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458956, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM
I20260811 14:35:56.949177 15120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:56.941709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34214 (local address 127.2.103.126:42013)
0811 14:35:56.941795 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:56.941796 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:56.941859 (+ 63us) server_negotiation.cc:407] Connection header received
0811 14:35:56.941944 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:56.941945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:56.941957 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:56.942024 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:56.942746 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.943263 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.943972 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.944144 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.944192 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.946516 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:56.946518 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:56.946519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:56.946527 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:56.947941 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:56.948300 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:56.948301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:56.948302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:56.948362 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:56.948574 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:56.948575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:56.948575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:56.948696 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:56.948804 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:56.948983 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 14:35:56.949071 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 14:35:56.950937 15034 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:35:56.952631 15034 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:35:56.959393 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:56.955555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33234 (local address 127.2.103.67:37791)
0811 14:35:56.955770 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:56.955775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:56.956112 (+ 337us) server_negotiation.cc:407] Connection header received
0811 14:35:56.956158 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:56.956160 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:56.956177 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:56.956271 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:56.956394 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.957070 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.958340 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.958933 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.959023 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.959106 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:56.959249 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 14:35:56.959292 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":101,"threads_started":1}
I20260811 14:35:56.959386 15916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:56.955438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:37791 (local address 127.0.0.1:33234)
0811 14:35:56.956021 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:56.956031 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:56.956084 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:56.956288 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:56.956289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:56.956291 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:56.956340 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:56.956341 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.957163 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.957164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:56.958236 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:56.958239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.959024 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.959025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:56.959107 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.959122 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 14:35:56.959229 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 14:35:56.959292 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":27,"thread_start_us":77,"threads_started":1}
I20260811 14:35:56.959759 15910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:56.955190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:46681 (local address 127.0.0.1:56200)
0811 14:35:56.955385 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:56.955426 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:56.955498 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:56.955999 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:56.956002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:56.956005 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:56.956076 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:56.956078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.957388 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.957389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:56.958561 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:56.958564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.959526 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.959527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:56.959595 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.959598 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:35:56.959632 (+ 34us) client_negotiation.cc:240] Negotiation successful
0811 14:35:56.959678 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":3,"thread_start_us":69,"threads_started":1}
I20260811 14:35:56.959908 15914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:56.955191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:35599 (local address 127.0.0.1:48048)
0811 14:35:56.955876 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:56.955906 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:56.956139 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:56.956404 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:56.956405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:56.956407 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:56.956451 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:56.956452 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.957642 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.957643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:56.958719 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:56.958721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.959677 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.959678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:56.959728 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.959731 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:35:56.959810 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 14:35:56.959853 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":41,"thread_start_us":80,"threads_started":1}
I20260811 14:35:56.960147 15919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:56.955547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56200 (local address 127.2.103.66:46681)
0811 14:35:56.955808 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:56.955812 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:56.955825 (+ 13us) server_negotiation.cc:407] Connection header received
0811 14:35:56.955870 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:56.955872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:56.955891 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:56.955994 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:56.956149 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.957284 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.958687 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.959405 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.959869 (+ 464us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.959954 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:56.959984 (+ 30us) server_negotiation.cc:299] Negotiation successful
0811 14:35:56.960036 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":120,"threads_started":1}
I20260811 14:35:56.960551 15915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:56.955314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48048 (local address 127.2.103.65:35599)
0811 14:35:56.955822 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:56.955828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:56.955972 (+ 144us) server_negotiation.cc:407] Connection header received
0811 14:35:56.956238 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:56.956241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:56.956257 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:56.956311 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:56.956526 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.957553 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.958807 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:56.959505 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:56.959546 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:56.959619 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:56.960360 (+ 741us) server_negotiation.cc:299] Negotiation successful
0811 14:35:56.960435 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":126,"threads_started":1}
I20260811 14:35:56.960464 15733 tablet_service.cc:1467] Processing CreateTablet for tablet b321c4f7b16f421ab5d3323b9a3fc439 (DEFAULT_TABLE table=test-table [id=ef9bf48057c1444fbfff1737934f2410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:56.960909 15733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b321c4f7b16f421ab5d3323b9a3fc439. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:56.960865 15503 tablet_service.cc:1467] Processing CreateTablet for tablet b321c4f7b16f421ab5d3323b9a3fc439 (DEFAULT_TABLE table=test-table [id=ef9bf48057c1444fbfff1737934f2410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:56.961190 15503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b321c4f7b16f421ab5d3323b9a3fc439. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:56.961269 15256 tablet_service.cc:1467] Processing CreateTablet for tablet b321c4f7b16f421ab5d3323b9a3fc439 (DEFAULT_TABLE table=test-table [id=ef9bf48057c1444fbfff1737934f2410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:56.961587 15256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b321c4f7b16f421ab5d3323b9a3fc439. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:56.964510 15920 tablet_bootstrap.cc:492] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa: Bootstrap starting.
I20260811 14:35:56.965005 15922 tablet_bootstrap.cc:492] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa: Bootstrap starting.
I20260811 14:35:56.965341 15920 tablet_bootstrap.cc:654] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:56.965675 15922 tablet_bootstrap.cc:654] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:56.965705 15923 tablet_bootstrap.cc:492] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83: Bootstrap starting.
I20260811 14:35:56.965909 15920 log.cc:824] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:56.966029 15922 log.cc:824] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:56.966648 15923 tablet_bootstrap.cc:654] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:56.966861 15920 tablet_bootstrap.cc:492] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa: No bootstrap required, opened a new log
I20260811 14:35:56.966931 15922 tablet_bootstrap.cc:492] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa: No bootstrap required, opened a new log
I20260811 14:35:56.966949 15920 ts_tablet_manager.cc:1397] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:35:56.967011 15922 ts_tablet_manager.cc:1397] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:35:56.967001 15923 log.cc:824] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:56.967787 15923 tablet_bootstrap.cc:492] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83: No bootstrap required, opened a new log
I20260811 14:35:56.967864 15923 ts_tablet_manager.cc:1397] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:35:56.968648 15920 raft_consensus.cc:348] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } }
I20260811 14:35:56.968811 15920 raft_consensus.cc:374] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:56.968856 15920 raft_consensus.cc:729] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 448de06301414a92bcc89b462b503faa, State: Initialized, Role: FOLLOWER
I20260811 14:35:56.968972 15920 consensus_queue.cc:260] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } }
I20260811 14:35:56.969282 15616 heartbeater.cc:503] Master 127.2.103.126:42013 was elected leader, sending a full tablet report...
I20260811 14:35:56.969182 15922 raft_consensus.cc:348] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } }
I20260811 14:35:56.969511 15922 raft_consensus.cc:374] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:56.969547 15922 raft_consensus.cc:729] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef83f76be0f497a81c571f8def5ecaa, State: Initialized, Role: FOLLOWER
I20260811 14:35:56.969676 15922 consensus_queue.cc:260] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } }
I20260811 14:35:56.969282 15920 ts_tablet_manager.cc:1428] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 14:35:56.969965 15922 ts_tablet_manager.cc:1428] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:35:56.970078 15867 heartbeater.cc:503] Master 127.2.103.126:42013 was elected leader, sending a full tablet report...
I20260811 14:35:56.969949 15923 raft_consensus.cc:348] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } }
I20260811 14:35:56.970441 15923 raft_consensus.cc:374] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:56.970501 15923 raft_consensus.cc:729] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95fe30de2e3341b19420c5e84a242b83, State: Initialized, Role: FOLLOWER
I20260811 14:35:56.970667 15923 consensus_queue.cc:260] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } }
I20260811 14:35:56.970954 15923 ts_tablet_manager.cc:1428] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:35:56.971026 15379 heartbeater.cc:503] Master 127.2.103.126:42013 was elected leader, sending a full tablet report...
W20260811 14:35:56.992173 15619 tablet.cc:2369] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:35:57.049181 15936 raft_consensus.cc:482] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:35:57.049346 15936 raft_consensus.cc:504] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } }
I20260811 14:35:57.049888 15936 leader_election.cc:288] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95fe30de2e3341b19420c5e84a242b83 (127.2.103.65:35599), 448de06301414a92bcc89b462b503faa (127.2.103.66:46681)
I20260811 14:35:57.054162 15915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:57.050246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:55895 (local address 127.2.103.65:35599)
0811 14:35:57.050298 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:57.050300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:57.050749 (+ 449us) server_negotiation.cc:407] Connection header received
0811 14:35:57.050823 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:57.050825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:57.050836 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:57.050897 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:57.051143 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.051786 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.053161 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.053720 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.053808 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.053817 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:57.054039 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 14:35:57.054084 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 14:35:57.054162 15919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:57.050315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:59787 (local address 127.2.103.66:46681)
0811 14:35:57.050362 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:57.050365 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:57.050374 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:35:57.050407 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:57.050408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:57.050433 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:57.050491 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:57.050857 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.051593 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.053164 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.053734 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.053808 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.053818 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:57.054041 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 14:35:57.054085 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:35:57.054169 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:57.050135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:46681 (local address 127.2.103.67:59787)
0811 14:35:57.050187 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:57.050199 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:57.050250 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:57.050600 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:57.050601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:57.050604 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:57.050754 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:57.050757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.051692 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.051693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:57.053050 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:57.053053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.053836 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.053837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:57.053943 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.053950 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 14:35:57.054017 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 14:35:57.054064 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":11}
I20260811 14:35:57.054181 15941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:57.050135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:35599 (local address 127.2.103.67:55895)
0811 14:35:57.050507 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:57.050520 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:57.050748 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:57.050974 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:57.050975 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:57.050977 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:57.051049 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:57.051050 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.051884 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.051885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:57.053050 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:57.053054 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.053836 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.053837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:57.053943 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.053950 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 14:35:57.054017 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 14:35:57.054063 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":183,"thread_start_us":126,"threads_started":1}
I20260811 14:35:57.054751 15535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b321c4f7b16f421ab5d3323b9a3fc439" candidate_uuid: "8ef83f76be0f497a81c571f8def5ecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448de06301414a92bcc89b462b503faa" is_pre_election: true
I20260811 14:35:57.054970 15535 raft_consensus.cc:2454] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef83f76be0f497a81c571f8def5ecaa in term 0.
I20260811 14:35:57.054955 15298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b321c4f7b16f421ab5d3323b9a3fc439" candidate_uuid: "8ef83f76be0f497a81c571f8def5ecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95fe30de2e3341b19420c5e84a242b83" is_pre_election: true
I20260811 14:35:57.055135 15298 raft_consensus.cc:2454] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef83f76be0f497a81c571f8def5ecaa in term 0.
I20260811 14:35:57.055291 15655 leader_election.cc:302] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 448de06301414a92bcc89b462b503faa, 8ef83f76be0f497a81c571f8def5ecaa; no voters:
I20260811 14:35:57.055436 15936 raft_consensus.cc:2781] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:35:57.055507 15936 raft_consensus.cc:482] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:35:57.055526 15936 raft_consensus.cc:3037] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:57.056476 15936 raft_consensus.cc:504] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } }
I20260811 14:35:57.056638 15936 leader_election.cc:288] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [CANDIDATE]: Term 1 election: Requested vote from peers 95fe30de2e3341b19420c5e84a242b83 (127.2.103.65:35599), 448de06301414a92bcc89b462b503faa (127.2.103.66:46681)
I20260811 14:35:57.056820 15535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b321c4f7b16f421ab5d3323b9a3fc439" candidate_uuid: "8ef83f76be0f497a81c571f8def5ecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448de06301414a92bcc89b462b503faa"
I20260811 14:35:57.056808 15298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b321c4f7b16f421ab5d3323b9a3fc439" candidate_uuid: "8ef83f76be0f497a81c571f8def5ecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95fe30de2e3341b19420c5e84a242b83"
I20260811 14:35:57.056905 15535 raft_consensus.cc:3037] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:57.056905 15298 raft_consensus.cc:3037] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:57.057847 15535 raft_consensus.cc:2454] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef83f76be0f497a81c571f8def5ecaa in term 1.
I20260811 14:35:57.057847 15298 raft_consensus.cc:2454] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef83f76be0f497a81c571f8def5ecaa in term 1.
I20260811 14:35:57.058049 15656 leader_election.cc:302] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef83f76be0f497a81c571f8def5ecaa, 95fe30de2e3341b19420c5e84a242b83; no voters:
I20260811 14:35:57.058192 15936 raft_consensus.cc:2781] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:57.058399 15936 raft_consensus.cc:686] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [term 1 LEADER]: Becoming Leader. State: Replica: 8ef83f76be0f497a81c571f8def5ecaa, State: Running, Role: LEADER
I20260811 14:35:57.058516 15936 consensus_queue.cc:237] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } }
I20260811 14:35:57.059557 15034 catalog_manager.cc:5495] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ef83f76be0f497a81c571f8def5ecaa (127.2.103.67). New cstate: current_term: 1 leader_uuid: "8ef83f76be0f497a81c571f8def5ecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } health_report { overall_health: UNKNOWN } } }
I20260811 14:35:57.078454 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:57.075931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33240 (local address 127.2.103.67:37791)
0811 14:35:57.075989 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:57.075992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:57.076002 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:35:57.076044 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:57.076045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:57.076063 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:57.076144 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:57.076313 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.076801 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.077603 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.077801 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.077899 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.078087 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:57.078207 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:57.078298 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:57.078339 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 14:35:57.078363 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 14:35:57.080562 15298 raft_consensus.cc:1261] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83 [term 1 FOLLOWER]: Refusing update from remote peer 8ef83f76be0f497a81c571f8def5ecaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:57.080562 15535 raft_consensus.cc:1261] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa [term 1 FOLLOWER]: Refusing update from remote peer 8ef83f76be0f497a81c571f8def5ecaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:57.080997 15936 consensus_queue.cc:1035] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:57.081203 15942 consensus_queue.cc:1035] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:57.086939 15034 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:57.087065 15034 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:57.087893 15034 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25: Sending DeleteTablet for 3 replicas of tablet b321c4f7b16f421ab5d3323b9a3fc439
I20260811 14:35:57.088265 15503 tablet_service.cc:1514] Processing DeleteTablet for tablet b321c4f7b16f421ab5d3323b9a3fc439 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:57 UTC) from {username='oryx', principal='oryx/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:56200
I20260811 14:35:57.088277 15256 tablet_service.cc:1514] Processing DeleteTablet for tablet b321c4f7b16f421ab5d3323b9a3fc439 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:57 UTC) from {username='oryx', principal='oryx/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:48048
I20260811 14:35:57.088485 15034 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:35:57.088670 15034 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:35:57.088963 15969 tablet_replica.cc:330] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83: stopping tablet replica
I20260811 14:35:57.088961 15733 tablet_service.cc:1514] Processing DeleteTablet for tablet b321c4f7b16f421ab5d3323b9a3fc439 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:57 UTC) from {username='oryx', principal='oryx/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:33234
I20260811 14:35:57.089075 15969 raft_consensus.cc:2229] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:57.089182 15969 raft_consensus.cc:2258] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:57.089349 15967 tablet_replica.cc:330] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa: stopping tablet replica
I20260811 14:35:57.089462 15967 raft_consensus.cc:2229] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:57.089829 15967 raft_consensus.cc:2258] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:57.090229 15733 tablet_service.cc:1467] Processing CreateTablet for tablet f371601b7d884fa2b4d3403804847946 (DEFAULT_TABLE table=test-table [id=4a9110bd24f247f98a4706d6e96604f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:57.090251 15256 tablet_service.cc:1467] Processing CreateTablet for tablet f371601b7d884fa2b4d3403804847946 (DEFAULT_TABLE table=test-table [id=4a9110bd24f247f98a4706d6e96604f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:57.090306 15969 ts_tablet_manager.cc:1905] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:57.090378 15256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f371601b7d884fa2b4d3403804847946. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:57.090385 15733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f371601b7d884fa2b4d3403804847946. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:57.091893 15923 tablet_bootstrap.cc:492] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83: Bootstrap starting.
I20260811 14:35:57.091887 15922 tablet_bootstrap.cc:492] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa: Bootstrap starting.
I20260811 14:35:57.092389 15923 tablet_bootstrap.cc:654] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:57.092397 15922 tablet_bootstrap.cc:654] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:57.092718 15967 ts_tablet_manager.cc:1905] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:57.092778 15969 ts_tablet_manager.cc:1918] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:35:57.092840 15969 log.cc:1196] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-0/wal/wals/b321c4f7b16f421ab5d3323b9a3fc439
I20260811 14:35:57.093137 15969 ts_tablet_manager.cc:1939] T b321c4f7b16f421ab5d3323b9a3fc439 P 95fe30de2e3341b19420c5e84a242b83: Deleting consensus metadata
I20260811 14:35:57.093127 15503 tablet_service.cc:1467] Processing CreateTablet for tablet f371601b7d884fa2b4d3403804847946 (DEFAULT_TABLE table=test-table [id=4a9110bd24f247f98a4706d6e96604f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:57.093222 15503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f371601b7d884fa2b4d3403804847946. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:57.093596 15015 catalog_manager.cc:4841] TS 95fe30de2e3341b19420c5e84a242b83 (127.2.103.65:35599): tablet b321c4f7b16f421ab5d3323b9a3fc439 (table test-table [id=ef9bf48057c1444fbfff1737934f2410]) successfully deleted
I20260811 14:35:57.094436 15967 ts_tablet_manager.cc:1918] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:35:57.094506 15967 log.cc:1196] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/b321c4f7b16f421ab5d3323b9a3fc439
I20260811 14:35:57.094525 15920 tablet_bootstrap.cc:492] T f371601b7d884fa2b4d3403804847946 P 448de06301414a92bcc89b462b503faa: Bootstrap starting.
I20260811 14:35:57.094789 15967 ts_tablet_manager.cc:1939] T b321c4f7b16f421ab5d3323b9a3fc439 P 448de06301414a92bcc89b462b503faa: Deleting consensus metadata
I20260811 14:35:57.095008 15920 tablet_bootstrap.cc:654] T f371601b7d884fa2b4d3403804847946 P 448de06301414a92bcc89b462b503faa: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:57.095224 15971 tablet_replica.cc:330] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa: stopping tablet replica
I20260811 14:35:57.095284 15014 catalog_manager.cc:4841] TS 448de06301414a92bcc89b462b503faa (127.2.103.66:46681): tablet b321c4f7b16f421ab5d3323b9a3fc439 (table test-table [id=ef9bf48057c1444fbfff1737934f2410]) successfully deleted
I20260811 14:35:57.095345 15971 raft_consensus.cc:2229] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:35:57.095490 15971 raft_consensus.cc:2258] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:57.095855 15920 tablet_bootstrap.cc:492] T f371601b7d884fa2b4d3403804847946 P 448de06301414a92bcc89b462b503faa: No bootstrap required, opened a new log
I20260811 14:35:57.095933 15920 ts_tablet_manager.cc:1397] T f371601b7d884fa2b4d3403804847946 P 448de06301414a92bcc89b462b503faa: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:57.096016 15971 ts_tablet_manager.cc:1905] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:57.096112 15920 raft_consensus.cc:348] T f371601b7d884fa2b4d3403804847946 P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.096201 15920 raft_consensus.cc:374] T f371601b7d884fa2b4d3403804847946 P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:57.096233 15920 raft_consensus.cc:729] T f371601b7d884fa2b4d3403804847946 P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 448de06301414a92bcc89b462b503faa, State: Initialized, Role: FOLLOWER
I20260811 14:35:57.096284 15920 consensus_queue.cc:260] T f371601b7d884fa2b4d3403804847946 P 448de06301414a92bcc89b462b503faa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.096371 15920 ts_tablet_manager.cc:1428] T f371601b7d884fa2b4d3403804847946 P 448de06301414a92bcc89b462b503faa: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:57.097193 15922 tablet_bootstrap.cc:492] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa: No bootstrap required, opened a new log
I20260811 14:35:57.097246 15922 ts_tablet_manager.cc:1397] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260811 14:35:57.097424 15922 raft_consensus.cc:348] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.097506 15922 raft_consensus.cc:374] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:57.097534 15922 raft_consensus.cc:729] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef83f76be0f497a81c571f8def5ecaa, State: Initialized, Role: FOLLOWER
I20260811 14:35:57.097585 15922 consensus_queue.cc:260] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.097627 15971 ts_tablet_manager.cc:1918] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:35:57.097687 15922 ts_tablet_manager.cc:1428] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:57.097687 15971 log.cc:1196] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/b321c4f7b16f421ab5d3323b9a3fc439
I20260811 14:35:57.097999 15971 ts_tablet_manager.cc:1939] T b321c4f7b16f421ab5d3323b9a3fc439 P 8ef83f76be0f497a81c571f8def5ecaa: Deleting consensus metadata
I20260811 14:35:57.098357 15015 catalog_manager.cc:4841] TS 8ef83f76be0f497a81c571f8def5ecaa (127.2.103.67:37791): tablet b321c4f7b16f421ab5d3323b9a3fc439 (table test-table [id=ef9bf48057c1444fbfff1737934f2410]) successfully deleted
I20260811 14:35:57.098853 15923 tablet_bootstrap.cc:492] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83: No bootstrap required, opened a new log
I20260811 14:35:57.098915 15923 ts_tablet_manager.cc:1397] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260811 14:35:57.099048 15923 raft_consensus.cc:348] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.099128 15923 raft_consensus.cc:374] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:57.099159 15923 raft_consensus.cc:729] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95fe30de2e3341b19420c5e84a242b83, State: Initialized, Role: FOLLOWER
I20260811 14:35:57.099205 15923 consensus_queue.cc:260] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.099298 15923 ts_tablet_manager.cc:1428] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20260811 14:35:57.159981 15868 tablet.cc:2369] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:35:57.338836 15380 tablet.cc:2369] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:35:57.403362 15937 raft_consensus.cc:482] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:35:57.403473 15937 raft_consensus.cc:504] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.403810 15937 leader_election.cc:288] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 448de06301414a92bcc89b462b503faa (127.2.103.66:46681), 8ef83f76be0f497a81c571f8def5ecaa (127.2.103.67:37791)
I20260811 14:35:57.407027 15919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:57.404123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:36651 (local address 127.2.103.66:46681)
0811 14:35:57.404179 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:57.404181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:57.404282 (+ 101us) server_negotiation.cc:407] Connection header received
0811 14:35:57.404399 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:57.404400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:57.404433 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:57.404494 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:57.404689 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.405172 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.406206 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.406566 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.406689 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.406699 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:57.406919 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 14:35:57.406967 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 14:35:57.407027 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:57.404165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:58575 (local address 127.2.103.67:37791)
0811 14:35:57.404217 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:57.404219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:57.404232 (+ 13us) server_negotiation.cc:407] Connection header received
0811 14:35:57.404356 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:57.404358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:57.404373 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:57.404435 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:35:57.404616 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.405118 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.406204 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.406573 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.406689 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.406700 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:57.406898 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 14:35:57.406956 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 14:35:57.407028 15978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:57.404039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:46681 (local address 127.2.103.65:36651)
0811 14:35:57.404198 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:57.404209 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:57.404323 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:57.404512 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:57.404514 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:57.404525 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:57.404616 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:57.404618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.405266 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.405267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:57.406097 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:57.406100 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.406746 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.406747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:57.406806 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.406809 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:35:57.406890 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 14:35:57.406930 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":17,"thread_start_us":76,"threads_started":1}
I20260811 14:35:57.407044 15977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:57.404031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:37791 (local address 127.2.103.65:58575)
0811 14:35:57.404178 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:57.404189 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:57.404251 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:57.404462 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:57.404463 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:57.404465 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:35:57.404534 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:57.404536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.405207 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.405209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:57.406098 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:57.406101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.406703 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.406704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:57.406783 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.406787 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:35:57.406878 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 14:35:57.406924 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":67,"thread_start_us":83,"threads_started":1}
I20260811 14:35:57.407330 15535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f371601b7d884fa2b4d3403804847946" candidate_uuid: "95fe30de2e3341b19420c5e84a242b83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448de06301414a92bcc89b462b503faa" is_pre_election: true
I20260811 14:35:57.407449 15535 raft_consensus.cc:2454] T f371601b7d884fa2b4d3403804847946 P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fe30de2e3341b19420c5e84a242b83 in term 0.
I20260811 14:35:57.407507 15770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f371601b7d884fa2b4d3403804847946" candidate_uuid: "95fe30de2e3341b19420c5e84a242b83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef83f76be0f497a81c571f8def5ecaa" is_pre_election: true
I20260811 14:35:57.407644 15770 raft_consensus.cc:2454] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fe30de2e3341b19420c5e84a242b83 in term 0.
I20260811 14:35:57.407634 15182 leader_election.cc:302] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 448de06301414a92bcc89b462b503faa, 95fe30de2e3341b19420c5e84a242b83; no voters:
I20260811 14:35:57.407788 15937 raft_consensus.cc:2781] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:35:57.407846 15937 raft_consensus.cc:482] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:35:57.407871 15937 raft_consensus.cc:3037] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:57.408591 15937 raft_consensus.cc:504] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.408736 15937 leader_election.cc:288] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [CANDIDATE]: Term 1 election: Requested vote from peers 448de06301414a92bcc89b462b503faa (127.2.103.66:46681), 8ef83f76be0f497a81c571f8def5ecaa (127.2.103.67:37791)
I20260811 14:35:57.408885 15535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f371601b7d884fa2b4d3403804847946" candidate_uuid: "95fe30de2e3341b19420c5e84a242b83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448de06301414a92bcc89b462b503faa"
I20260811 14:35:57.408937 15770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f371601b7d884fa2b4d3403804847946" candidate_uuid: "95fe30de2e3341b19420c5e84a242b83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef83f76be0f497a81c571f8def5ecaa"
I20260811 14:35:57.408983 15535 raft_consensus.cc:3037] T f371601b7d884fa2b4d3403804847946 P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:57.409018 15770 raft_consensus.cc:3037] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:57.409615 15535 raft_consensus.cc:2454] T f371601b7d884fa2b4d3403804847946 P 448de06301414a92bcc89b462b503faa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fe30de2e3341b19420c5e84a242b83 in term 1.
I20260811 14:35:57.409775 15770 raft_consensus.cc:2454] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fe30de2e3341b19420c5e84a242b83 in term 1.
I20260811 14:35:57.409803 15182 leader_election.cc:302] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 448de06301414a92bcc89b462b503faa, 95fe30de2e3341b19420c5e84a242b83; no voters:
I20260811 14:35:57.409883 15937 raft_consensus.cc:2781] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:57.409998 15937 raft_consensus.cc:686] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [term 1 LEADER]: Becoming Leader. State: Replica: 95fe30de2e3341b19420c5e84a242b83, State: Running, Role: LEADER
I20260811 14:35:57.410072 15937 consensus_queue.cc:237] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.410894 15034 catalog_manager.cc:5495] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95fe30de2e3341b19420c5e84a242b83 (127.2.103.65). New cstate: current_term: 1 leader_uuid: "95fe30de2e3341b19420c5e84a242b83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } health_report { overall_health: UNKNOWN } } }
Aug 11 14:35:57 dist-test-slave-twwt krb5kdc[14961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458956, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.126@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM
I20260811 14:35:57.478997 15994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:57.471389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42013 (local address 127.0.0.1:34216)
0811 14:35:57.471559 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:57.471575 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:57.471662 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:57.471844 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:57.471845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:57.472218 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:57.472270 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:57.472273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.472948 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.472949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:57.473496 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:57.473498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.473834 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.473835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:57.473881 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.474522 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:57.474527 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:57.476109 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:57.477854 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:57.477861 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:57.477863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:57.478071 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:57.478313 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:57.478314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:57.478314 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:57.478370 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:57.478603 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:57.478664 (+ 61us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:57.478717 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 14:35:57.478774 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 14:35:57.478876 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":72,"threads_started":1}
I20260811 14:35:57.479048 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:57.471460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34216 (local address 127.2.103.126:42013)
0811 14:35:57.471625 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:57.471627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:57.471638 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:35:57.471725 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:57.471725 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:57.471736 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:57.471798 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:57.472360 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.472875 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.473568 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.473760 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.473811 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.476204 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:57.476206 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:57.476206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:57.476213 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:57.477778 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:57.478162 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:57.478163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:57.478163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:57.478232 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:57.478429 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:57.478429 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:57.478430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:57.478549 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:57.478655 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:57.478803 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 14:35:57.478934 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260811 14:35:57.481407 15034 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:34216:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 14:35:57.481616 15034 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:35:57.483191 15503 tablet_service.cc:1467] Processing CreateTablet for tablet 70f0bda86fd3403eb14efe79d68857dc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a76c4c4e9f214fe6b1b52310c0ccd5ff]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 14:35:57.483181 15733 tablet_service.cc:1467] Processing CreateTablet for tablet 70f0bda86fd3403eb14efe79d68857dc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a76c4c4e9f214fe6b1b52310c0ccd5ff]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 14:35:57.483196 15256 tablet_service.cc:1467] Processing CreateTablet for tablet 70f0bda86fd3403eb14efe79d68857dc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a76c4c4e9f214fe6b1b52310c0ccd5ff]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 14:35:57.483384 15503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70f0bda86fd3403eb14efe79d68857dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:57.483385 15733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70f0bda86fd3403eb14efe79d68857dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:57.483385 15256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70f0bda86fd3403eb14efe79d68857dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:57.484665 15920 tablet_bootstrap.cc:492] T 70f0bda86fd3403eb14efe79d68857dc P 448de06301414a92bcc89b462b503faa: Bootstrap starting.
I20260811 14:35:57.484975 15922 tablet_bootstrap.cc:492] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa: Bootstrap starting.
I20260811 14:35:57.485154 15920 tablet_bootstrap.cc:654] T 70f0bda86fd3403eb14efe79d68857dc P 448de06301414a92bcc89b462b503faa: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:57.485335 15923 tablet_bootstrap.cc:492] T 70f0bda86fd3403eb14efe79d68857dc P 95fe30de2e3341b19420c5e84a242b83: Bootstrap starting.
I20260811 14:35:57.485548 15922 tablet_bootstrap.cc:654] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:57.485800 15923 tablet_bootstrap.cc:654] T 70f0bda86fd3403eb14efe79d68857dc P 95fe30de2e3341b19420c5e84a242b83: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:57.486112 15922 tablet_bootstrap.cc:492] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa: No bootstrap required, opened a new log
I20260811 14:35:57.486109 15920 tablet_bootstrap.cc:492] T 70f0bda86fd3403eb14efe79d68857dc P 448de06301414a92bcc89b462b503faa: No bootstrap required, opened a new log
I20260811 14:35:57.486167 15920 ts_tablet_manager.cc:1397] T 70f0bda86fd3403eb14efe79d68857dc P 448de06301414a92bcc89b462b503faa: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260811 14:35:57.486184 15922 ts_tablet_manager.cc:1397] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:57.486317 15920 raft_consensus.cc:348] T 70f0bda86fd3403eb14efe79d68857dc P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.486310 15922 raft_consensus.cc:348] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.486393 15920 raft_consensus.cc:374] T 70f0bda86fd3403eb14efe79d68857dc P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:57.486393 15922 raft_consensus.cc:374] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:57.486416 15920 raft_consensus.cc:729] T 70f0bda86fd3403eb14efe79d68857dc P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 448de06301414a92bcc89b462b503faa, State: Initialized, Role: FOLLOWER
I20260811 14:35:57.486429 15922 raft_consensus.cc:729] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef83f76be0f497a81c571f8def5ecaa, State: Initialized, Role: FOLLOWER
I20260811 14:35:57.486467 15920 consensus_queue.cc:260] T 70f0bda86fd3403eb14efe79d68857dc P 448de06301414a92bcc89b462b503faa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.486476 15922 consensus_queue.cc:260] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.486572 15922 ts_tablet_manager.cc:1428] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:57.486588 15920 ts_tablet_manager.cc:1428] T 70f0bda86fd3403eb14efe79d68857dc P 448de06301414a92bcc89b462b503faa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:57.486658 15923 tablet_bootstrap.cc:492] T 70f0bda86fd3403eb14efe79d68857dc P 95fe30de2e3341b19420c5e84a242b83: No bootstrap required, opened a new log
I20260811 14:35:57.486714 15923 ts_tablet_manager.cc:1397] T 70f0bda86fd3403eb14efe79d68857dc P 95fe30de2e3341b19420c5e84a242b83: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:57.486734 15962 tablet_replica.cc:439] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } } }
I20260811 14:35:57.486758 15933 tablet_replica.cc:439] T 70f0bda86fd3403eb14efe79d68857dc P 448de06301414a92bcc89b462b503faa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } } }
I20260811 14:35:57.486812 15962 tablet_replica.cc:442] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:35:57.486825 15933 tablet_replica.cc:442] T 70f0bda86fd3403eb14efe79d68857dc P 448de06301414a92bcc89b462b503faa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:35:57.486867 15923 raft_consensus.cc:348] T 70f0bda86fd3403eb14efe79d68857dc P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.486951 15923 raft_consensus.cc:374] T 70f0bda86fd3403eb14efe79d68857dc P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:57.486981 15923 raft_consensus.cc:729] T 70f0bda86fd3403eb14efe79d68857dc P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95fe30de2e3341b19420c5e84a242b83, State: Initialized, Role: FOLLOWER
I20260811 14:35:57.487035 15923 consensus_queue.cc:260] T 70f0bda86fd3403eb14efe79d68857dc P 95fe30de2e3341b19420c5e84a242b83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.487133 15923 ts_tablet_manager.cc:1428] T 70f0bda86fd3403eb14efe79d68857dc P 95fe30de2e3341b19420c5e84a242b83: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:57.487195 15937 tablet_replica.cc:439] T 70f0bda86fd3403eb14efe79d68857dc P 95fe30de2e3341b19420c5e84a242b83: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } } }
I20260811 14:35:57.487306 15937 tablet_replica.cc:442] T 70f0bda86fd3403eb14efe79d68857dc P 95fe30de2e3341b19420c5e84a242b83: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:35:57.578027 15962 raft_consensus.cc:482] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:35:57.578133 15962 raft_consensus.cc:504] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.578303 15962 leader_election.cc:288] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95fe30de2e3341b19420c5e84a242b83 (127.2.103.65:35599), 448de06301414a92bcc89b462b503faa (127.2.103.66:46681)
I20260811 14:35:57.578506 15298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70f0bda86fd3403eb14efe79d68857dc" candidate_uuid: "8ef83f76be0f497a81c571f8def5ecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95fe30de2e3341b19420c5e84a242b83" is_pre_election: true
I20260811 14:35:57.578673 15298 raft_consensus.cc:2454] T 70f0bda86fd3403eb14efe79d68857dc P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef83f76be0f497a81c571f8def5ecaa in term 0.
I20260811 14:35:57.578634 15535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70f0bda86fd3403eb14efe79d68857dc" candidate_uuid: "8ef83f76be0f497a81c571f8def5ecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448de06301414a92bcc89b462b503faa" is_pre_election: true
I20260811 14:35:57.578776 15535 raft_consensus.cc:2454] T 70f0bda86fd3403eb14efe79d68857dc P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef83f76be0f497a81c571f8def5ecaa in term 0.
I20260811 14:35:57.578872 15656 leader_election.cc:302] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef83f76be0f497a81c571f8def5ecaa, 95fe30de2e3341b19420c5e84a242b83; no voters:
I20260811 14:35:57.578975 15962 raft_consensus.cc:2781] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:35:57.579003 15962 raft_consensus.cc:482] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:35:57.579030 15962 raft_consensus.cc:3037] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:57.579794 15962 raft_consensus.cc:504] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.579944 15962 leader_election.cc:288] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa [CANDIDATE]: Term 1 election: Requested vote from peers 95fe30de2e3341b19420c5e84a242b83 (127.2.103.65:35599), 448de06301414a92bcc89b462b503faa (127.2.103.66:46681)
I20260811 14:35:57.580117 15535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70f0bda86fd3403eb14efe79d68857dc" candidate_uuid: "8ef83f76be0f497a81c571f8def5ecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448de06301414a92bcc89b462b503faa"
I20260811 14:35:57.580098 15298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70f0bda86fd3403eb14efe79d68857dc" candidate_uuid: "8ef83f76be0f497a81c571f8def5ecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95fe30de2e3341b19420c5e84a242b83"
I20260811 14:35:57.580192 15535 raft_consensus.cc:3037] T 70f0bda86fd3403eb14efe79d68857dc P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:57.580192 15298 raft_consensus.cc:3037] T 70f0bda86fd3403eb14efe79d68857dc P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:57.580998 15298 raft_consensus.cc:2454] T 70f0bda86fd3403eb14efe79d68857dc P 95fe30de2e3341b19420c5e84a242b83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef83f76be0f497a81c571f8def5ecaa in term 1.
I20260811 14:35:57.580998 15535 raft_consensus.cc:2454] T 70f0bda86fd3403eb14efe79d68857dc P 448de06301414a92bcc89b462b503faa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef83f76be0f497a81c571f8def5ecaa in term 1.
I20260811 14:35:57.581223 15655 leader_election.cc:302] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 448de06301414a92bcc89b462b503faa, 8ef83f76be0f497a81c571f8def5ecaa, 95fe30de2e3341b19420c5e84a242b83; no voters:
I20260811 14:35:57.581332 15962 raft_consensus.cc:2781] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:57.581404 15962 raft_consensus.cc:686] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa [term 1 LEADER]: Becoming Leader. State: Replica: 8ef83f76be0f497a81c571f8def5ecaa, State: Running, Role: LEADER
I20260811 14:35:57.581463 15962 consensus_queue.cc:237] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.581741 15942 tablet_replica.cc:439] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa: TxnStatusTablet state changed. Reason: New leader 8ef83f76be0f497a81c571f8def5ecaa. Latest consensus state: current_term: 1 leader_uuid: "8ef83f76be0f497a81c571f8def5ecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } } }
I20260811 14:35:57.581843 15942 tablet_replica.cc:442] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa: This TxnStatusTablet replica's current role is: LEADER
I20260811 14:35:57.582091 16005 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 14:35:57.582237 16005 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 14:35:57.582147 15030 catalog_manager.cc:5495] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ef83f76be0f497a81c571f8def5ecaa (127.2.103.67). New cstate: current_term: 1 leader_uuid: "8ef83f76be0f497a81c571f8def5ecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } health_report { overall_health: HEALTHY } } }
I20260811 14:35:57.605991 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:57.603253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33252 (local address 127.2.103.67:37791)
0811 14:35:57.603296 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:57.603298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:57.603319 (+ 21us) server_negotiation.cc:407] Connection header received
0811 14:35:57.603412 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:57.603413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:57.603423 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:57.603469 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:57.603597 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.604068 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.605243 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.605450 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.605520 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.605726 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:57.605757 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:57.605797 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:57.605904 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 14:35:57.605926 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:35:57.606053 15994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:57.603172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:37791 (local address 127.0.0.1:33252)
0811 14:35:57.603261 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:57.603275 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:57.603331 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:57.603480 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:57.603481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:57.603482 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:35:57.603538 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:57.603540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.604140 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.604141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:57.605120 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:57.605124 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.605538 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.605539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:57.605612 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.605636 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:35:57.605820 (+ 184us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:35:57.605831 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 14:35:57.605903 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 14:35:57.605964 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 14:35:57.607321 15535 raft_consensus.cc:1261] T 70f0bda86fd3403eb14efe79d68857dc P 448de06301414a92bcc89b462b503faa [term 1 FOLLOWER]: Refusing update from remote peer 8ef83f76be0f497a81c571f8def5ecaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:57.607409 15298 raft_consensus.cc:1261] T 70f0bda86fd3403eb14efe79d68857dc P 95fe30de2e3341b19420c5e84a242b83 [term 1 FOLLOWER]: Refusing update from remote peer 8ef83f76be0f497a81c571f8def5ecaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:57.607585 15962 consensus_queue.cc:1035] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:57.607688 15942 consensus_queue.cc:1035] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:57.608165 15937 tablet_replica.cc:439] T 70f0bda86fd3403eb14efe79d68857dc P 95fe30de2e3341b19420c5e84a242b83: TxnStatusTablet state changed. Reason: New leader 8ef83f76be0f497a81c571f8def5ecaa. Latest consensus state: current_term: 1 leader_uuid: "8ef83f76be0f497a81c571f8def5ecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } } }
I20260811 14:35:57.608253 15937 tablet_replica.cc:442] T 70f0bda86fd3403eb14efe79d68857dc P 95fe30de2e3341b19420c5e84a242b83: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:35:57.608260 15933 tablet_replica.cc:439] T 70f0bda86fd3403eb14efe79d68857dc P 448de06301414a92bcc89b462b503faa: TxnStatusTablet state changed. Reason: New leader 8ef83f76be0f497a81c571f8def5ecaa. Latest consensus state: current_term: 1 leader_uuid: "8ef83f76be0f497a81c571f8def5ecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } } }
I20260811 14:35:57.608350 15933 tablet_replica.cc:442] T 70f0bda86fd3403eb14efe79d68857dc P 448de06301414a92bcc89b462b503faa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:35:57.609143 15942 tablet_replica.cc:439] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8ef83f76be0f497a81c571f8def5ecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } } }
I20260811 14:35:57.609243 15942 tablet_replica.cc:442] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa: This TxnStatusTablet replica's current role is: LEADER
I20260811 14:35:57.609280 15962 tablet_replica.cc:439] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8ef83f76be0f497a81c571f8def5ecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } } }
I20260811 14:35:57.609357 15962 tablet_replica.cc:442] T 70f0bda86fd3403eb14efe79d68857dc P 8ef83f76be0f497a81c571f8def5ecaa: This TxnStatusTablet replica's current role is: LEADER
I20260811 14:35:57.609386 15937 tablet_replica.cc:439] T 70f0bda86fd3403eb14efe79d68857dc P 95fe30de2e3341b19420c5e84a242b83: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8ef83f76be0f497a81c571f8def5ecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } } }
I20260811 14:35:57.609481 15937 tablet_replica.cc:442] T 70f0bda86fd3403eb14efe79d68857dc P 95fe30de2e3341b19420c5e84a242b83: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:35:57.609768 15933 tablet_replica.cc:439] T 70f0bda86fd3403eb14efe79d68857dc P 448de06301414a92bcc89b462b503faa: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8ef83f76be0f497a81c571f8def5ecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } } }
I20260811 14:35:57.609856 15933 tablet_replica.cc:442] T 70f0bda86fd3403eb14efe79d68857dc P 448de06301414a92bcc89b462b503faa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:35:57.609949 16011 mvcc.cc:205] Tried to move back new op lower bound from 7317335890357772288 to 7317335890254041088. Current Snapshot: MvccSnapshot[applied={T|T < 7317335890357772288}]
I20260811 14:35:57.610211 16010 mvcc.cc:205] Tried to move back new op lower bound from 7317335890357772288 to 7317335890254041088. Current Snapshot: MvccSnapshot[applied={T|T < 7317335890357772288}]
I20260811 14:35:57.610685 16014 mvcc.cc:205] Tried to move back new op lower bound from 7317335890357772288 to 7317335890254041088. Current Snapshot: MvccSnapshot[applied={T|T < 7317335890357772288 or (T in {7317335890357772288})}]
I20260811 14:35:57.614884 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:57.611899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48058 (local address 127.2.103.65:35599)
0811 14:35:57.611991 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:57.611994 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:57.612004 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:35:57.612057 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:57.612057 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:57.612075 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:57.612149 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:57.612292 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.612849 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.613977 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.614208 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.614272 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.614455 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:57.614565 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:57.614705 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:57.614726 (+ 21us) server_negotiation.cc:299] Negotiation successful
0811 14:35:57.614751 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73,"thread_start_us":51,"threads_started":1}
I20260811 14:35:57.620477 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:57.617705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:36621 (local address 127.2.103.67:37791)
0811 14:35:57.617757 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:57.617759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:57.617861 (+ 102us) server_negotiation.cc:407] Connection header received
0811 14:35:57.617895 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:57.617896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:57.617908 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:57.617977 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:57.618064 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.618827 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.619620 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.619922 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.619996 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.620231 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:57.620266 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:57.620319 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:57.620372 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 14:35:57.620395 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 14:35:57.620507 16033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:57.617638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:37791 (local address 127.2.103.65:36621)
0811 14:35:57.617788 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:57.617803 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:57.617857 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:57.617978 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:57.617979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:57.617980 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:35:57.618027 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:57.618028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.618906 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.618907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:57.619533 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:57.619535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.620054 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.620056 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:57.620115 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.620133 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:35:57.620327 (+ 194us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:35:57.620329 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:35:57.620368 (+ 39us) client_negotiation.cc:240] Negotiation successful
0811 14:35:57.620406 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
I20260811 14:35:57.622956 15535 raft_consensus.cc:1261] T f371601b7d884fa2b4d3403804847946 P 448de06301414a92bcc89b462b503faa [term 1 FOLLOWER]: Refusing update from remote peer 95fe30de2e3341b19420c5e84a242b83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:57.623014 15770 raft_consensus.cc:1261] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa [term 1 FOLLOWER]: Refusing update from remote peer 95fe30de2e3341b19420c5e84a242b83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:57.623519 15979 consensus_queue.cc:1035] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:57.623689 15937 consensus_queue.cc:1035] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:57.632975 16039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:57.630003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:35599 (local address 127.2.103.67:49145)
0811 14:35:57.630168 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:57.630186 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:57.630254 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:57.630392 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:57.630393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:57.630394 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:35:57.630471 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:57.630473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.631177 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.631178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:57.632023 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:57.632027 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.632493 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.632493 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:57.632547 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.632563 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:35:57.632787 (+ 224us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:35:57.632790 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:35:57.632845 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 14:35:57.632899 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":92,"threads_started":1}
I20260811 14:35:57.632956 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:57.630189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:49145 (local address 127.2.103.65:35599)
0811 14:35:57.630251 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:57.630253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:57.630269 (+ 16us) server_negotiation.cc:407] Connection header received
0811 14:35:57.630316 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:57.630317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:57.630326 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:57.630388 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:57.630548 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.631087 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.632122 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.632406 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.632442 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.632662 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:57.632698 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:57.632786 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:57.632851 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 14:35:57.632874 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 14:35:57.646706 15031 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:57.646838 15031 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:57.647549 15031 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25: Sending DeleteTablet for 3 replicas of tablet f371601b7d884fa2b4d3403804847946
I20260811 14:35:57.647846 15503 tablet_service.cc:1514] Processing DeleteTablet for tablet f371601b7d884fa2b4d3403804847946 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:57 UTC) from {username='oryx', principal='oryx/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:56200
I20260811 14:35:57.647888 15256 tablet_service.cc:1514] Processing DeleteTablet for tablet f371601b7d884fa2b4d3403804847946 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:57 UTC) from {username='oryx', principal='oryx/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:48048
I20260811 14:35:57.648002 15733 tablet_service.cc:1514] Processing DeleteTablet for tablet f371601b7d884fa2b4d3403804847946 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:57 UTC) from {username='oryx', principal='oryx/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:33234
I20260811 14:35:57.648237 16046 tablet_replica.cc:330] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa: stopping tablet replica
I20260811 14:35:57.648314 16046 raft_consensus.cc:2229] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:57.648352 16046 raft_consensus.cc:2258] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:57.651889 16046 ts_tablet_manager.cc:1905] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:57.653844 16046 ts_tablet_manager.cc:1918] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:35:57.653929 16046 log.cc:1196] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/f371601b7d884fa2b4d3403804847946
I20260811 14:35:57.654153 16046 ts_tablet_manager.cc:1939] T f371601b7d884fa2b4d3403804847946 P 8ef83f76be0f497a81c571f8def5ecaa: Deleting consensus metadata
I20260811 14:35:57.654640 15015 catalog_manager.cc:4841] TS 8ef83f76be0f497a81c571f8def5ecaa (127.2.103.67:37791): tablet f371601b7d884fa2b4d3403804847946 (table test-table [id=4a9110bd24f247f98a4706d6e96604f2]) successfully deleted
I20260811 14:35:57.655946 16044 tablet_replica.cc:330] T f371601b7d884fa2b4d3403804847946 P 448de06301414a92bcc89b462b503faa: stopping tablet replica
I20260811 14:35:57.656069 16044 raft_consensus.cc:2229] T f371601b7d884fa2b4d3403804847946 P 448de06301414a92bcc89b462b503faa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:57.656126 16044 raft_consensus.cc:2258] T f371601b7d884fa2b4d3403804847946 P 448de06301414a92bcc89b462b503faa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:57.662293 16044 ts_tablet_manager.cc:1905] T f371601b7d884fa2b4d3403804847946 P 448de06301414a92bcc89b462b503faa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:57.664229 16044 ts_tablet_manager.cc:1918] T f371601b7d884fa2b4d3403804847946 P 448de06301414a92bcc89b462b503faa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:35:57.664400 16044 log.cc:1196] T f371601b7d884fa2b4d3403804847946 P 448de06301414a92bcc89b462b503faa: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/f371601b7d884fa2b4d3403804847946
I20260811 14:35:57.664709 16044 ts_tablet_manager.cc:1939] T f371601b7d884fa2b4d3403804847946 P 448de06301414a92bcc89b462b503faa: Deleting consensus metadata
I20260811 14:35:57.665143 16042 tablet_replica.cc:330] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83: stopping tablet replica
I20260811 14:35:57.665210 15014 catalog_manager.cc:4841] TS 448de06301414a92bcc89b462b503faa (127.2.103.66:46681): tablet f371601b7d884fa2b4d3403804847946 (table test-table [id=4a9110bd24f247f98a4706d6e96604f2]) successfully deleted
I20260811 14:35:57.665343 16042 raft_consensus.cc:2229] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:35:57.665519 16042 raft_consensus.cc:2258] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:57.665872 16042 ts_tablet_manager.cc:1905] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 14:35:57 dist-test-slave-twwt krb5kdc[14961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260811 14:35:57.667912 16042 ts_tablet_manager.cc:1918] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:35:57.668072 16042 log.cc:1196] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-0/wal/wals/f371601b7d884fa2b4d3403804847946
I20260811 14:35:57.668396 16042 ts_tablet_manager.cc:1939] T f371601b7d884fa2b4d3403804847946 P 95fe30de2e3341b19420c5e84a242b83: Deleting consensus metadata
I20260811 14:35:57.668818 15015 catalog_manager.cc:4841] TS 95fe30de2e3341b19420c5e84a242b83 (127.2.103.65:35599): tablet f371601b7d884fa2b4d3403804847946 (table test-table [id=4a9110bd24f247f98a4706d6e96604f2]) successfully deleted
Aug 11 14:35:57 dist-test-slave-twwt krb5kdc[14961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:35:57 dist-test-slave-twwt krb5kdc[14961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 14:35:57.686549 15995 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:57.675877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34232 (local address 127.2.103.126:42013)
0811 14:35:57.675933 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:57.675935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:57.676082 (+ 147us) server_negotiation.cc:407] Connection header received
0811 14:35:57.676153 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:57.676155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:57.676167 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:57.676247 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:57.677039 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.677857 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.679073 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.679393 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.679490 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.686475 (+ 6985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34232: BlockingRecv error: recv got EOF from 127.0.0.1:34232 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 11 14:35:57 dist-test-slave-twwt krb5kdc[14961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM
I20260811 14:35:57.696626 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:57.687514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34244 (local address 127.2.103.126:42013)
0811 14:35:57.687554 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:57.687556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:57.687692 (+ 136us) server_negotiation.cc:407] Connection header received
0811 14:35:57.687750 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:57.687751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:57.687764 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:57.687819 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:57.688413 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.689195 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.690225 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.690556 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.690657 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.693435 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:57.693437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:57.693437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:57.693445 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:57.695223 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:57.695669 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:57.695670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:57.695671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:57.695735 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:57.695962 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:57.695963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:57.695964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:57.696106 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:57.696239 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:57.696418 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 14:35:57.696508 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:35:57.698446 15031 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34244:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:35:57.698798 15031 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:35:57.700201 15503 tablet_service.cc:1467] Processing CreateTablet for tablet fcaf4770d7364e63bbae1efd20f60853 (DEFAULT_TABLE table=test-table [id=56908ce5a5e74ddbaa72c236140293fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:57.700229 15256 tablet_service.cc:1467] Processing CreateTablet for tablet fcaf4770d7364e63bbae1efd20f60853 (DEFAULT_TABLE table=test-table [id=56908ce5a5e74ddbaa72c236140293fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:57.700366 15503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcaf4770d7364e63bbae1efd20f60853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:57.700369 15256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcaf4770d7364e63bbae1efd20f60853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:57.701701 15920 tablet_bootstrap.cc:492] T fcaf4770d7364e63bbae1efd20f60853 P 448de06301414a92bcc89b462b503faa: Bootstrap starting.
I20260811 14:35:57.701735 15923 tablet_bootstrap.cc:492] T fcaf4770d7364e63bbae1efd20f60853 P 95fe30de2e3341b19420c5e84a242b83: Bootstrap starting.
I20260811 14:35:57.701766 15733 tablet_service.cc:1467] Processing CreateTablet for tablet fcaf4770d7364e63bbae1efd20f60853 (DEFAULT_TABLE table=test-table [id=56908ce5a5e74ddbaa72c236140293fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:57.701890 15733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcaf4770d7364e63bbae1efd20f60853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:57.702162 15920 tablet_bootstrap.cc:654] T fcaf4770d7364e63bbae1efd20f60853 P 448de06301414a92bcc89b462b503faa: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:57.702188 15923 tablet_bootstrap.cc:654] T fcaf4770d7364e63bbae1efd20f60853 P 95fe30de2e3341b19420c5e84a242b83: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:57.702960 15923 tablet_bootstrap.cc:492] T fcaf4770d7364e63bbae1efd20f60853 P 95fe30de2e3341b19420c5e84a242b83: No bootstrap required, opened a new log
I20260811 14:35:57.703009 15923 ts_tablet_manager.cc:1397] T fcaf4770d7364e63bbae1efd20f60853 P 95fe30de2e3341b19420c5e84a242b83: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:57.703151 15923 raft_consensus.cc:348] T fcaf4770d7364e63bbae1efd20f60853 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.703214 15923 raft_consensus.cc:374] T fcaf4770d7364e63bbae1efd20f60853 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:57.703233 15923 raft_consensus.cc:729] T fcaf4770d7364e63bbae1efd20f60853 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95fe30de2e3341b19420c5e84a242b83, State: Initialized, Role: FOLLOWER
I20260811 14:35:57.703274 15923 consensus_queue.cc:260] T fcaf4770d7364e63bbae1efd20f60853 P 95fe30de2e3341b19420c5e84a242b83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.703349 15923 ts_tablet_manager.cc:1428] T fcaf4770d7364e63bbae1efd20f60853 P 95fe30de2e3341b19420c5e84a242b83: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:57.703365 15922 tablet_bootstrap.cc:492] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa: Bootstrap starting.
I20260811 14:35:57.703711 15922 tablet_bootstrap.cc:654] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:57.704610 15922 tablet_bootstrap.cc:492] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa: No bootstrap required, opened a new log
I20260811 14:35:57.704667 15922 ts_tablet_manager.cc:1397] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 14:35:57.704818 15922 raft_consensus.cc:348] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.704890 15922 raft_consensus.cc:374] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:57.704910 15922 raft_consensus.cc:729] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef83f76be0f497a81c571f8def5ecaa, State: Initialized, Role: FOLLOWER
I20260811 14:35:57.704957 15922 consensus_queue.cc:260] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.705047 15922 ts_tablet_manager.cc:1428] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 14:35:57.707079 15920 tablet_bootstrap.cc:492] T fcaf4770d7364e63bbae1efd20f60853 P 448de06301414a92bcc89b462b503faa: No bootstrap required, opened a new log
I20260811 14:35:57.707130 15920 ts_tablet_manager.cc:1397] T fcaf4770d7364e63bbae1efd20f60853 P 448de06301414a92bcc89b462b503faa: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260811 14:35:57.707257 15920 raft_consensus.cc:348] T fcaf4770d7364e63bbae1efd20f60853 P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.707314 15920 raft_consensus.cc:374] T fcaf4770d7364e63bbae1efd20f60853 P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:57.707329 15920 raft_consensus.cc:729] T fcaf4770d7364e63bbae1efd20f60853 P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 448de06301414a92bcc89b462b503faa, State: Initialized, Role: FOLLOWER
I20260811 14:35:57.707391 15920 consensus_queue.cc:260] T fcaf4770d7364e63bbae1efd20f60853 P 448de06301414a92bcc89b462b503faa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.707512 15920 ts_tablet_manager.cc:1428] T fcaf4770d7364e63bbae1efd20f60853 P 448de06301414a92bcc89b462b503faa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:57.892937 16020 raft_consensus.cc:482] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:35:57.893072 16020 raft_consensus.cc:504] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.893290 16020 leader_election.cc:288] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95fe30de2e3341b19420c5e84a242b83 (127.2.103.65:35599), 448de06301414a92bcc89b462b503faa (127.2.103.66:46681)
I20260811 14:35:57.893520 15298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcaf4770d7364e63bbae1efd20f60853" candidate_uuid: "8ef83f76be0f497a81c571f8def5ecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95fe30de2e3341b19420c5e84a242b83" is_pre_election: true
I20260811 14:35:57.893537 15535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcaf4770d7364e63bbae1efd20f60853" candidate_uuid: "8ef83f76be0f497a81c571f8def5ecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448de06301414a92bcc89b462b503faa" is_pre_election: true
I20260811 14:35:57.893664 15298 raft_consensus.cc:2454] T fcaf4770d7364e63bbae1efd20f60853 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef83f76be0f497a81c571f8def5ecaa in term 0.
I20260811 14:35:57.893679 15535 raft_consensus.cc:2454] T fcaf4770d7364e63bbae1efd20f60853 P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef83f76be0f497a81c571f8def5ecaa in term 0.
I20260811 14:35:57.893860 15656 leader_election.cc:302] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef83f76be0f497a81c571f8def5ecaa, 95fe30de2e3341b19420c5e84a242b83; no voters:
I20260811 14:35:57.893962 16020 raft_consensus.cc:2781] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:35:57.894021 16020 raft_consensus.cc:482] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:35:57.894037 16020 raft_consensus.cc:3037] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:57.894804 16020 raft_consensus.cc:504] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.894949 16020 leader_election.cc:288] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [CANDIDATE]: Term 1 election: Requested vote from peers 95fe30de2e3341b19420c5e84a242b83 (127.2.103.65:35599), 448de06301414a92bcc89b462b503faa (127.2.103.66:46681)
I20260811 14:35:57.895089 15298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcaf4770d7364e63bbae1efd20f60853" candidate_uuid: "8ef83f76be0f497a81c571f8def5ecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95fe30de2e3341b19420c5e84a242b83"
I20260811 14:35:57.895104 15535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcaf4770d7364e63bbae1efd20f60853" candidate_uuid: "8ef83f76be0f497a81c571f8def5ecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448de06301414a92bcc89b462b503faa"
I20260811 14:35:57.895171 15298 raft_consensus.cc:3037] T fcaf4770d7364e63bbae1efd20f60853 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:57.895174 15535 raft_consensus.cc:3037] T fcaf4770d7364e63bbae1efd20f60853 P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:57.895993 15298 raft_consensus.cc:2454] T fcaf4770d7364e63bbae1efd20f60853 P 95fe30de2e3341b19420c5e84a242b83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef83f76be0f497a81c571f8def5ecaa in term 1.
I20260811 14:35:57.895993 15535 raft_consensus.cc:2454] T fcaf4770d7364e63bbae1efd20f60853 P 448de06301414a92bcc89b462b503faa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef83f76be0f497a81c571f8def5ecaa in term 1.
I20260811 14:35:57.896175 15655 leader_election.cc:302] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 448de06301414a92bcc89b462b503faa, 8ef83f76be0f497a81c571f8def5ecaa; no voters:
I20260811 14:35:57.896278 16020 raft_consensus.cc:2781] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:57.896327 16020 raft_consensus.cc:686] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [term 1 LEADER]: Becoming Leader. State: Replica: 8ef83f76be0f497a81c571f8def5ecaa, State: Running, Role: LEADER
I20260811 14:35:57.896381 16020 consensus_queue.cc:237] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.897025 15031 catalog_manager.cc:5495] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ef83f76be0f497a81c571f8def5ecaa (127.2.103.67). New cstate: current_term: 1 leader_uuid: "8ef83f76be0f497a81c571f8def5ecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } health_report { overall_health: HEALTHY } } }
I20260811 14:35:57.907845 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:57.905368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33254 (local address 127.2.103.67:37791)
0811 14:35:57.905446 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:57.905448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:57.905457 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:35:57.905482 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:57.905483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:57.905501 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:57.905562 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:57.905710 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.906184 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.907036 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:57.907286 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:57.907352 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:57.907541 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:57.907581 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:57.907647 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:57.907708 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 14:35:57.907773 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 14:35:57.908759 15535 raft_consensus.cc:1261] T fcaf4770d7364e63bbae1efd20f60853 P 448de06301414a92bcc89b462b503faa [term 1 FOLLOWER]: Refusing update from remote peer 8ef83f76be0f497a81c571f8def5ecaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:57.908759 15298 raft_consensus.cc:1261] T fcaf4770d7364e63bbae1efd20f60853 P 95fe30de2e3341b19420c5e84a242b83 [term 1 FOLLOWER]: Refusing update from remote peer 8ef83f76be0f497a81c571f8def5ecaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:57.908985 16020 consensus_queue.cc:1035] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:57.909144 16020 consensus_queue.cc:1035] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:57.912205 15030 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:57.912305 15030 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:57.913013 15030 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25: Sending DeleteTablet for 3 replicas of tablet fcaf4770d7364e63bbae1efd20f60853
I20260811 14:35:57.913267 15733 tablet_service.cc:1514] Processing DeleteTablet for tablet fcaf4770d7364e63bbae1efd20f60853 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:57 UTC) from {username='oryx', principal='oryx/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:33234
I20260811 14:35:57.913281 15503 tablet_service.cc:1514] Processing DeleteTablet for tablet fcaf4770d7364e63bbae1efd20f60853 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:57 UTC) from {username='oryx', principal='oryx/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:56200
I20260811 14:35:57.913374 16044 tablet_replica.cc:330] T fcaf4770d7364e63bbae1efd20f60853 P 448de06301414a92bcc89b462b503faa: stopping tablet replica
I20260811 14:35:57.913393 16046 tablet_replica.cc:330] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa: stopping tablet replica
I20260811 14:35:57.913430 16044 raft_consensus.cc:2229] T fcaf4770d7364e63bbae1efd20f60853 P 448de06301414a92bcc89b462b503faa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:57.913452 16046 raft_consensus.cc:2229] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:35:57.913470 16044 raft_consensus.cc:2258] T fcaf4770d7364e63bbae1efd20f60853 P 448de06301414a92bcc89b462b503faa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:57.913516 16046 raft_consensus.cc:2258] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:57.913549 15256 tablet_service.cc:1514] Processing DeleteTablet for tablet fcaf4770d7364e63bbae1efd20f60853 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:57 UTC) from {username='oryx', principal='oryx/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:48048
I20260811 14:35:57.913591 15030 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34244:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 14:35:57.913648 16042 tablet_replica.cc:330] T fcaf4770d7364e63bbae1efd20f60853 P 95fe30de2e3341b19420c5e84a242b83: stopping tablet replica
I20260811 14:35:57.913717 16042 raft_consensus.cc:2229] T fcaf4770d7364e63bbae1efd20f60853 P 95fe30de2e3341b19420c5e84a242b83 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 14:35:57.913722 15030 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:35:57.913767 16042 raft_consensus.cc:2258] T fcaf4770d7364e63bbae1efd20f60853 P 95fe30de2e3341b19420c5e84a242b83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:57.914111 16042 ts_tablet_manager.cc:1905] T fcaf4770d7364e63bbae1efd20f60853 P 95fe30de2e3341b19420c5e84a242b83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:57.914248 16046 ts_tablet_manager.cc:1905] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:57.914772 16044 ts_tablet_manager.cc:1905] T fcaf4770d7364e63bbae1efd20f60853 P 448de06301414a92bcc89b462b503faa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:57.915627 16042 ts_tablet_manager.cc:1918] T fcaf4770d7364e63bbae1efd20f60853 P 95fe30de2e3341b19420c5e84a242b83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:35:57.915706 16042 log.cc:1196] T fcaf4770d7364e63bbae1efd20f60853 P 95fe30de2e3341b19420c5e84a242b83: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-0/wal/wals/fcaf4770d7364e63bbae1efd20f60853
I20260811 14:35:57.915913 16046 ts_tablet_manager.cc:1918] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:35:57.915941 16042 ts_tablet_manager.cc:1939] T fcaf4770d7364e63bbae1efd20f60853 P 95fe30de2e3341b19420c5e84a242b83: Deleting consensus metadata
I20260811 14:35:57.915978 16046 log.cc:1196] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/fcaf4770d7364e63bbae1efd20f60853
I20260811 14:35:57.916195 16046 ts_tablet_manager.cc:1939] T fcaf4770d7364e63bbae1efd20f60853 P 8ef83f76be0f497a81c571f8def5ecaa: Deleting consensus metadata
I20260811 14:35:57.916297 15015 catalog_manager.cc:4841] TS 95fe30de2e3341b19420c5e84a242b83 (127.2.103.65:35599): tablet fcaf4770d7364e63bbae1efd20f60853 (table test-table [id=56908ce5a5e74ddbaa72c236140293fd]) successfully deleted
I20260811 14:35:57.916359 16044 ts_tablet_manager.cc:1918] T fcaf4770d7364e63bbae1efd20f60853 P 448de06301414a92bcc89b462b503faa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:35:57.916409 16044 log.cc:1196] T fcaf4770d7364e63bbae1efd20f60853 P 448de06301414a92bcc89b462b503faa: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/fcaf4770d7364e63bbae1efd20f60853
I20260811 14:35:57.916512 15015 catalog_manager.cc:4841] TS 8ef83f76be0f497a81c571f8def5ecaa (127.2.103.67:37791): tablet fcaf4770d7364e63bbae1efd20f60853 (table test-table [id=56908ce5a5e74ddbaa72c236140293fd]) successfully deleted
I20260811 14:35:57.916632 16044 ts_tablet_manager.cc:1939] T fcaf4770d7364e63bbae1efd20f60853 P 448de06301414a92bcc89b462b503faa: Deleting consensus metadata
I20260811 14:35:57.916980 15014 catalog_manager.cc:4841] TS 448de06301414a92bcc89b462b503faa (127.2.103.66:46681): tablet fcaf4770d7364e63bbae1efd20f60853 (table test-table [id=56908ce5a5e74ddbaa72c236140293fd]) successfully deleted
I20260811 14:35:57.917312 15503 tablet_service.cc:1467] Processing CreateTablet for tablet c8abdd1045c74bc0b8a074c95be03984 (DEFAULT_TABLE table=test-table [id=1b503721d0994ae480aab7a3743fa13c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:57.917312 15256 tablet_service.cc:1467] Processing CreateTablet for tablet c8abdd1045c74bc0b8a074c95be03984 (DEFAULT_TABLE table=test-table [id=1b503721d0994ae480aab7a3743fa13c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:57.917323 15733 tablet_service.cc:1467] Processing CreateTablet for tablet c8abdd1045c74bc0b8a074c95be03984 (DEFAULT_TABLE table=test-table [id=1b503721d0994ae480aab7a3743fa13c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:35:57.917434 15256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8abdd1045c74bc0b8a074c95be03984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:57.917433 15503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8abdd1045c74bc0b8a074c95be03984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:57.917460 15733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8abdd1045c74bc0b8a074c95be03984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:57.918699 15920 tablet_bootstrap.cc:492] T c8abdd1045c74bc0b8a074c95be03984 P 448de06301414a92bcc89b462b503faa: Bootstrap starting.
I20260811 14:35:57.918808 15922 tablet_bootstrap.cc:492] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa: Bootstrap starting.
I20260811 14:35:57.918850 15923 tablet_bootstrap.cc:492] T c8abdd1045c74bc0b8a074c95be03984 P 95fe30de2e3341b19420c5e84a242b83: Bootstrap starting.
I20260811 14:35:57.919126 15922 tablet_bootstrap.cc:654] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:57.919153 15920 tablet_bootstrap.cc:654] T c8abdd1045c74bc0b8a074c95be03984 P 448de06301414a92bcc89b462b503faa: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:57.919165 15923 tablet_bootstrap.cc:654] T c8abdd1045c74bc0b8a074c95be03984 P 95fe30de2e3341b19420c5e84a242b83: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:57.919802 15920 tablet_bootstrap.cc:492] T c8abdd1045c74bc0b8a074c95be03984 P 448de06301414a92bcc89b462b503faa: No bootstrap required, opened a new log
I20260811 14:35:57.919843 15922 tablet_bootstrap.cc:492] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa: No bootstrap required, opened a new log
I20260811 14:35:57.919867 15920 ts_tablet_manager.cc:1397] T c8abdd1045c74bc0b8a074c95be03984 P 448de06301414a92bcc89b462b503faa: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:57.919876 15922 ts_tablet_manager.cc:1397] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 14:35:57.919975 15922 raft_consensus.cc:348] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.920037 15922 raft_consensus.cc:374] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:57.920051 15922 raft_consensus.cc:729] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef83f76be0f497a81c571f8def5ecaa, State: Initialized, Role: FOLLOWER
I20260811 14:35:57.920030 15920 raft_consensus.cc:348] T c8abdd1045c74bc0b8a074c95be03984 P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.920127 15920 raft_consensus.cc:374] T c8abdd1045c74bc0b8a074c95be03984 P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:57.920118 15922 consensus_queue.cc:260] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.920156 15920 raft_consensus.cc:729] T c8abdd1045c74bc0b8a074c95be03984 P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 448de06301414a92bcc89b462b503faa, State: Initialized, Role: FOLLOWER
I20260811 14:35:57.920202 15920 consensus_queue.cc:260] T c8abdd1045c74bc0b8a074c95be03984 P 448de06301414a92bcc89b462b503faa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.920213 15922 ts_tablet_manager.cc:1428] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:57.920295 15920 ts_tablet_manager.cc:1428] T c8abdd1045c74bc0b8a074c95be03984 P 448de06301414a92bcc89b462b503faa: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:35:57.920629 15923 tablet_bootstrap.cc:492] T c8abdd1045c74bc0b8a074c95be03984 P 95fe30de2e3341b19420c5e84a242b83: No bootstrap required, opened a new log
I20260811 14:35:57.920684 15923 ts_tablet_manager.cc:1397] T c8abdd1045c74bc0b8a074c95be03984 P 95fe30de2e3341b19420c5e84a242b83: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:35:57.920814 15923 raft_consensus.cc:348] T c8abdd1045c74bc0b8a074c95be03984 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.920884 15923 raft_consensus.cc:374] T c8abdd1045c74bc0b8a074c95be03984 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:57.920899 15923 raft_consensus.cc:729] T c8abdd1045c74bc0b8a074c95be03984 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95fe30de2e3341b19420c5e84a242b83, State: Initialized, Role: FOLLOWER
I20260811 14:35:57.920954 15923 consensus_queue.cc:260] T c8abdd1045c74bc0b8a074c95be03984 P 95fe30de2e3341b19420c5e84a242b83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:57.921047 15923 ts_tablet_manager.cc:1428] T c8abdd1045c74bc0b8a074c95be03984 P 95fe30de2e3341b19420c5e84a242b83: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:58.057976 15962 raft_consensus.cc:482] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:35:58.058100 15962 raft_consensus.cc:504] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:58.058311 15962 leader_election.cc:288] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95fe30de2e3341b19420c5e84a242b83 (127.2.103.65:35599), 448de06301414a92bcc89b462b503faa (127.2.103.66:46681)
I20260811 14:35:58.058571 15298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8abdd1045c74bc0b8a074c95be03984" candidate_uuid: "8ef83f76be0f497a81c571f8def5ecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95fe30de2e3341b19420c5e84a242b83" is_pre_election: true
I20260811 14:35:58.058707 15298 raft_consensus.cc:2454] T c8abdd1045c74bc0b8a074c95be03984 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef83f76be0f497a81c571f8def5ecaa in term 0.
I20260811 14:35:58.058707 15535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8abdd1045c74bc0b8a074c95be03984" candidate_uuid: "8ef83f76be0f497a81c571f8def5ecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448de06301414a92bcc89b462b503faa" is_pre_election: true
I20260811 14:35:58.058898 15535 raft_consensus.cc:2454] T c8abdd1045c74bc0b8a074c95be03984 P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef83f76be0f497a81c571f8def5ecaa in term 0.
I20260811 14:35:58.058939 15656 leader_election.cc:302] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef83f76be0f497a81c571f8def5ecaa, 95fe30de2e3341b19420c5e84a242b83; no voters:
I20260811 14:35:58.059073 15962 raft_consensus.cc:2781] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:35:58.059124 15962 raft_consensus.cc:482] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:35:58.059157 15962 raft_consensus.cc:3037] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:58.059914 15962 raft_consensus.cc:504] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:58.060078 15962 leader_election.cc:288] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [CANDIDATE]: Term 1 election: Requested vote from peers 95fe30de2e3341b19420c5e84a242b83 (127.2.103.65:35599), 448de06301414a92bcc89b462b503faa (127.2.103.66:46681)
I20260811 14:35:58.060261 15298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8abdd1045c74bc0b8a074c95be03984" candidate_uuid: "8ef83f76be0f497a81c571f8def5ecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95fe30de2e3341b19420c5e84a242b83"
I20260811 14:35:58.060267 15535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8abdd1045c74bc0b8a074c95be03984" candidate_uuid: "8ef83f76be0f497a81c571f8def5ecaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448de06301414a92bcc89b462b503faa"
I20260811 14:35:58.060331 15535 raft_consensus.cc:3037] T c8abdd1045c74bc0b8a074c95be03984 P 448de06301414a92bcc89b462b503faa [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:58.060352 15298 raft_consensus.cc:3037] T c8abdd1045c74bc0b8a074c95be03984 P 95fe30de2e3341b19420c5e84a242b83 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:58.061025 15535 raft_consensus.cc:2454] T c8abdd1045c74bc0b8a074c95be03984 P 448de06301414a92bcc89b462b503faa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef83f76be0f497a81c571f8def5ecaa in term 1.
I20260811 14:35:58.061034 15298 raft_consensus.cc:2454] T c8abdd1045c74bc0b8a074c95be03984 P 95fe30de2e3341b19420c5e84a242b83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef83f76be0f497a81c571f8def5ecaa in term 1.
I20260811 14:35:58.061226 15656 leader_election.cc:302] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef83f76be0f497a81c571f8def5ecaa, 95fe30de2e3341b19420c5e84a242b83; no voters:
I20260811 14:35:58.061336 15962 raft_consensus.cc:2781] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:58.061407 15962 raft_consensus.cc:686] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [term 1 LEADER]: Becoming Leader. State: Replica: 8ef83f76be0f497a81c571f8def5ecaa, State: Running, Role: LEADER
I20260811 14:35:58.061479 15962 consensus_queue.cc:237] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } }
I20260811 14:35:58.062187 15031 catalog_manager.cc:5495] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ef83f76be0f497a81c571f8def5ecaa (127.2.103.67). New cstate: current_term: 1 leader_uuid: "8ef83f76be0f497a81c571f8def5ecaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef83f76be0f497a81c571f8def5ecaa" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37791 } health_report { overall_health: HEALTHY } } }
I20260811 14:35:58.071492 16039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:58.068854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:37791 (local address 127.2.103.67:42397)
0811 14:35:58.068924 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:58.068941 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:58.069004 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:58.069151 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:58.069152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:58.069153 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:35:58.069211 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:58.069212 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:58.069990 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:58.069992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:58.070660 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:58.070663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:58.071042 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:58.071043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:58.071100 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:58.071106 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:35:58.071308 (+ 202us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:35:58.071310 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:35:58.071364 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 14:35:58.071414 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":5}
I20260811 14:35:58.071492 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:58.068915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:42397 (local address 127.2.103.67:37791)
0811 14:35:58.068957 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:58.068959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:58.068985 (+ 26us) server_negotiation.cc:407] Connection header received
0811 14:35:58.069073 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:58.069073 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:58.069085 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:58.069147 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:58.069308 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:58.069892 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:58.070735 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:58.070951 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:58.071042 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:58.071187 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:58.071225 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:58.071305 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:58.071366 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 14:35:58.071389 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":19}
I20260811 14:35:58.073565 15298 raft_consensus.cc:1261] T c8abdd1045c74bc0b8a074c95be03984 P 95fe30de2e3341b19420c5e84a242b83 [term 1 FOLLOWER]: Refusing update from remote peer 8ef83f76be0f497a81c571f8def5ecaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:58.073566 15535 raft_consensus.cc:1261] T c8abdd1045c74bc0b8a074c95be03984 P 448de06301414a92bcc89b462b503faa [term 1 FOLLOWER]: Refusing update from remote peer 8ef83f76be0f497a81c571f8def5ecaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:58.073809 15942 consensus_queue.cc:1035] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fe30de2e3341b19420c5e84a242b83" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:58.073944 15962 consensus_queue.cc:1035] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "448de06301414a92bcc89b462b503faa" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:58.084544 15031 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:58.084671 15031 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:35:58.085393 15031 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cbe4bde4366476ba2fc7f4cd3824c25: Sending DeleteTablet for 3 replicas of tablet c8abdd1045c74bc0b8a074c95be03984
I20260811 14:35:58.085660 15503 tablet_service.cc:1514] Processing DeleteTablet for tablet c8abdd1045c74bc0b8a074c95be03984 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:58 UTC) from {username='oryx', principal='oryx/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:56200
I20260811 14:35:58.085668 15733 tablet_service.cc:1514] Processing DeleteTablet for tablet c8abdd1045c74bc0b8a074c95be03984 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:58 UTC) from {username='oryx', principal='oryx/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:33234
I20260811 14:35:58.085721 15256 tablet_service.cc:1514] Processing DeleteTablet for tablet c8abdd1045c74bc0b8a074c95be03984 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:35:58 UTC) from {username='oryx', principal='oryx/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:48048
I20260811 14:35:58.085769 16046 tablet_replica.cc:330] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa: stopping tablet replica
I20260811 14:35:58.085817 16046 raft_consensus.cc:2229] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:35:58.085843 16044 tablet_replica.cc:330] T c8abdd1045c74bc0b8a074c95be03984 P 448de06301414a92bcc89b462b503faa: stopping tablet replica
I20260811 14:35:58.085875 16046 raft_consensus.cc:2258] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:58.085922 16044 raft_consensus.cc:2229] T c8abdd1045c74bc0b8a074c95be03984 P 448de06301414a92bcc89b462b503faa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:58.085979 16042 tablet_replica.cc:330] T c8abdd1045c74bc0b8a074c95be03984 P 95fe30de2e3341b19420c5e84a242b83: stopping tablet replica
I20260811 14:35:58.085969 16044 raft_consensus.cc:2258] T c8abdd1045c74bc0b8a074c95be03984 P 448de06301414a92bcc89b462b503faa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:58.086059 16042 raft_consensus.cc:2229] T c8abdd1045c74bc0b8a074c95be03984 P 95fe30de2e3341b19420c5e84a242b83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:35:58.086095 16042 raft_consensus.cc:2258] T c8abdd1045c74bc0b8a074c95be03984 P 95fe30de2e3341b19420c5e84a242b83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:35:58.086517 16042 ts_tablet_manager.cc:1905] T c8abdd1045c74bc0b8a074c95be03984 P 95fe30de2e3341b19420c5e84a242b83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:58.086555 16046 ts_tablet_manager.cc:1905] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:58.086809 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 15152
I20260811 14:35:58.091764 16046 ts_tablet_manager.cc:1918] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:35:58.091857 16046 log.cc:1196] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/c8abdd1045c74bc0b8a074c95be03984
I20260811 14:35:58.092082 16046 ts_tablet_manager.cc:1939] T c8abdd1045c74bc0b8a074c95be03984 P 8ef83f76be0f497a81c571f8def5ecaa: Deleting consensus metadata
I20260811 14:35:58.092499 15015 catalog_manager.cc:4841] TS 8ef83f76be0f497a81c571f8def5ecaa (127.2.103.67:37791): tablet c8abdd1045c74bc0b8a074c95be03984 (table test-table [id=1b503721d0994ae480aab7a3743fa13c]) successfully deleted
I20260811 14:35:58.092736 16044 ts_tablet_manager.cc:1905] T c8abdd1045c74bc0b8a074c95be03984 P 448de06301414a92bcc89b462b503faa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:35:58.095721 16044 ts_tablet_manager.cc:1918] T c8abdd1045c74bc0b8a074c95be03984 P 448de06301414a92bcc89b462b503faa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:35:58.095808 16044 log.cc:1196] T c8abdd1045c74bc0b8a074c95be03984 P 448de06301414a92bcc89b462b503faa: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/c8abdd1045c74bc0b8a074c95be03984
I20260811 14:35:58.096097 16044 ts_tablet_manager.cc:1939] T c8abdd1045c74bc0b8a074c95be03984 P 448de06301414a92bcc89b462b503faa: Deleting consensus metadata
I20260811 14:35:58.096513 15014 catalog_manager.cc:4841] TS 448de06301414a92bcc89b462b503faa (127.2.103.66:46681): tablet c8abdd1045c74bc0b8a074c95be03984 (table test-table [id=1b503721d0994ae480aab7a3743fa13c]) successfully deleted
W20260811 14:35:58.102910 15015 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.103.65:35599 (error 108)
I20260811 14:35:58.103611 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 15394
I20260811 14:35:58.103636 16089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:58.103399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:35599 (local address 127.0.0.1:48064)
0811 14:35:58.103507 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:58.103585 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.65:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":66,"threads_started":1}
W20260811 14:35:58.106127 15015 catalog_manager.cc:4584] TS 95fe30de2e3341b19420c5e84a242b83 (127.2.103.65:35599): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c8abdd1045c74bc0b8a074c95be03984: Network error: Client connection negotiation failed: client connection to 127.2.103.65:35599: connect: Connection refused (error 111)
I20260811 14:35:58.120496 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 15628
I20260811 14:35:58.137316 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 14989
2026-08-11T14:35:58Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3300 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:58 dist-test-slave-twwt krb5kdc[16093](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:58 dist-test-slave-twwt krb5kdc[16093](info): set up 2 sockets
Aug 11 14:35:58 dist-test-slave-twwt krb5kdc[16093](info): commencing operation
krb5kdc: starting...
W20260811 14:35:59.442013 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:35:59 dist-test-slave-twwt krb5kdc[16093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:35:59.573539 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:33265
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:43487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:33265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:59.662137 16119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:59.662361 16119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:59.662384 16119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:59.663534 16119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:59.663573 16119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:59.663587 16119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:59.663600 16119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:59.663612 16119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:59.665086 16119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:43487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:33265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:33265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:59.665292 16119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:59.665468 16119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:59.670406 16130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:59.670413 16127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:59.670423 16126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:59.670759 16119 server_base.cc:1034] running on GCE node
I20260811 14:35:59.671032 16119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:59.671339 16119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:59.672529 16119 hybrid_clock.cc:648] HybridClock initialized: now 1786458959672498 us; error 52 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T14:35:59Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1536 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:59 dist-test-slave-twwt krb5kdc[16136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:59 dist-test-slave-twwt krb5kdc[16136](info): set up 2 sockets
Aug 11 14:35:59 dist-test-slave-twwt krb5kdc[16136](info): commencing operation
krb5kdc: starting...
W20260811 14:36:00.948539 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:36:01 dist-test-slave-twwt krb5kdc[16136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:36:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:36:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:36:01.079712 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:42517
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:37253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:42517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:36:01.166452 16166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:36:01.166739 16166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:36:01.166781 16166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:36:01.167950 16166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:36:01.168023 16166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:36:01.168048 16166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:36:01.168081 16166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:36:01.168131 16166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:36:01.169658 16166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:37253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:42517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:42517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:36:01.169963 16166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:36:01.170207 16166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:36:01.175510 16173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:01.175529 16181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:01.175529 16172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:01.176007 16166 server_base.cc:1034] running on GCE node
I20260811 14:36:01.176290 16166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:36:01.176608 16166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:36:01.177805 16166 hybrid_clock.cc:648] HybridClock initialized: now 1786458961177787 us; error 49 us; skew 500 ppm
Aug 11 14:36:01 dist-test-slave-twwt krb5kdc[16136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458961, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:36:01.180563 16166 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:36:01.181180 16166 webserver.cc:460] Webserver started at http://127.2.103.126:39123/ using document root <none> and password file <none>
I20260811 14:36:01.181449 16166 fs_manager.cc:359] Metadata directory not provided
I20260811 14:36:01.181525 16166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:36:01.181689 16166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:36:01.182996 16166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "6ac44aec7c67497ba92abbc02ba87588"
format_stamp: "Formatted at 2026-08-11 14:36:01 on dist-test-slave-twwt"
server_key: "c012721ebd903b9821141d4b07323413"
server_key_iv: "9da005bd6ea53be0d0fd853c9fb58e21"
server_key_version: "encryptionkey@0"
I20260811 14:36:01.183403 16166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "6ac44aec7c67497ba92abbc02ba87588"
format_stamp: "Formatted at 2026-08-11 14:36:01 on dist-test-slave-twwt"
server_key: "c012721ebd903b9821141d4b07323413"
server_key_iv: "9da005bd6ea53be0d0fd853c9fb58e21"
server_key_version: "encryptionkey@0"
I20260811 14:36:01.184888 16166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:36:01.185745 16191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:01.185979 16166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:36:01.186067 16166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "6ac44aec7c67497ba92abbc02ba87588"
format_stamp: "Formatted at 2026-08-11 14:36:01 on dist-test-slave-twwt"
server_key: "c012721ebd903b9821141d4b07323413"
server_key_iv: "9da005bd6ea53be0d0fd853c9fb58e21"
server_key_version: "encryptionkey@0"
I20260811 14:36:01.186129 16166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:36:01.200243 16166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:36:01.202296 16166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:36:01.202425 16166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:36:01.207281 16166 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:42517
I20260811 14:36:01.207319 16278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:42517 every 8 connection(s)
I20260811 14:36:01.207770 16166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:36:01.208755 16280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:36:01.211210 16280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588: Bootstrap starting.
I20260811 14:36:01.211802 16280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588: Neither blocks nor log segments found. Creating new log.
I20260811 14:36:01.212064 16280 log.cc:824] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588: Log is configured to *not* fsync() on all Append() calls
I20260811 14:36:01.212770 16280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588: No bootstrap required, opened a new log
I20260811 14:36:01.214359 16280 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac44aec7c67497ba92abbc02ba87588" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42517 } }
I20260811 14:36:01.214512 16280 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:36:01.214536 16280 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ac44aec7c67497ba92abbc02ba87588, State: Initialized, Role: FOLLOWER
I20260811 14:36:01.214715 16280 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac44aec7c67497ba92abbc02ba87588" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42517 } }
I20260811 14:36:01.214785 16280 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:36:01.214807 16280 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:36:01.214838 16280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:36:01.215339 16280 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac44aec7c67497ba92abbc02ba87588" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42517 } }
I20260811 14:36:01.215435 16280 leader_election.cc:302] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ac44aec7c67497ba92abbc02ba87588; no voters:
I20260811 14:36:01.215564 16280 leader_election.cc:288] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:36:01.215664 16284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:36:01.215866 16280 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:36:01.216135 16284 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [term 1 LEADER]: Becoming Leader. State: Replica: 6ac44aec7c67497ba92abbc02ba87588, State: Running, Role: LEADER
I20260811 14:36:01.216318 16284 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac44aec7c67497ba92abbc02ba87588" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42517 } }
I20260811 14:36:01.216325 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 16166
I20260811 14:36:01.216466 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:36:01.216701 2461 external_mini_cluster.cc:1428] Setting key ea38583497ba11b20b3e37612d181e39
I20260811 14:36:01.216735 16286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ac44aec7c67497ba92abbc02ba87588. Latest consensus state: current_term: 1 leader_uuid: "6ac44aec7c67497ba92abbc02ba87588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac44aec7c67497ba92abbc02ba87588" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42517 } } }
I20260811 14:36:01.216830 16286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [sys.catalog]: This master's current role is: LEADER
I20260811 14:36:01.217141 16285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ac44aec7c67497ba92abbc02ba87588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac44aec7c67497ba92abbc02ba87588" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42517 } } }
I20260811 14:36:01.217250 16285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [sys.catalog]: This master's current role is: LEADER
W20260811 14:36:01.218005 16304 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:36:01.218092 16304 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:36:01.218182 16295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:36:01.218413 16295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:36:01.220216 16295 catalog_manager.cc:1347] Generated new cluster ID: cdbd0420492843a8bfa81d75798cc8cc
I20260811 14:36:01.220266 16295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:36:01 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:36:01.229295 16306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:01.217303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56278 (local address 127.2.103.126:42517)
0811 14:36:01.217714 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:01.217725 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:01.218036 (+ 311us) server_negotiation.cc:407] Connection header received
0811 14:36:01.218174 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:01.218182 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:01.218242 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:01.218395 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:01.219235 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:01.220555 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:01.221673 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:01.222285 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:01.222382 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:01.225468 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:36:01.225473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:36:01.225474 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:36:01.225488 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:36:01.227524 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:36:01.227967 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:36:01.227970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:36:01.227971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:36:01.228086 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:36:01.228308 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:36:01.228309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:36:01.228309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:36:01.228632 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:36:01.228776 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:01.228934 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 14:36:01.229070 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":74,"threads_started":1}
I20260811 14:36:01.243885 16295 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:36:01.244665 16295 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:36:01.249133 16295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588: Generated new TSK 0
I20260811 14:36:01.249351 16295 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
I20260811 14:36:01.290092 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:42517
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:37253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:36:01.400933 16315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:36:01.401171 16315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:36:01.401197 16315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:36:01.402451 16315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:36:01.402501 16315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:36:01.402560 16315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:36:01.404119 16315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:37253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:42517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:36:01.404381 16315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:36:01.404592 16315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:36:01.409751 16321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:01.409781 16320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:01.409751 16327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:01.410069 16315 server_base.cc:1034] running on GCE node
I20260811 14:36:01.410342 16315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:36:01.410679 16315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:36:01.411849 16315 hybrid_clock.cc:648] HybridClock initialized: now 1786458961411810 us; error 53 us; skew 500 ppm
Aug 11 14:36:01 dist-test-slave-twwt krb5kdc[16136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458961, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:36:01.419943 16315 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:36:01.420428 16315 webserver.cc:460] Webserver started at http://127.2.103.65:35561/ using document root <none> and password file <none>
I20260811 14:36:01.420670 16315 fs_manager.cc:359] Metadata directory not provided
I20260811 14:36:01.420740 16315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:36:01.420902 16315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:36:01.422014 16315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "172eeef9ca314d87b8b8e1888ace2917"
format_stamp: "Formatted at 2026-08-11 14:36:01 on dist-test-slave-twwt"
server_key: "f6eeeede693872cb54a5e371042038a2"
server_key_iv: "6c700f6f88d856c2da4da96d9037fafc"
server_key_version: "encryptionkey@0"
I20260811 14:36:01.422397 16315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "172eeef9ca314d87b8b8e1888ace2917"
format_stamp: "Formatted at 2026-08-11 14:36:01 on dist-test-slave-twwt"
server_key: "f6eeeede693872cb54a5e371042038a2"
server_key_iv: "6c700f6f88d856c2da4da96d9037fafc"
server_key_version: "encryptionkey@0"
I20260811 14:36:01.423898 16315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:36:01.424787 16339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:01.425014 16315 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:36:01.425124 16315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "172eeef9ca314d87b8b8e1888ace2917"
format_stamp: "Formatted at 2026-08-11 14:36:01 on dist-test-slave-twwt"
server_key: "f6eeeede693872cb54a5e371042038a2"
server_key_iv: "6c700f6f88d856c2da4da96d9037fafc"
server_key_version: "encryptionkey@0"
I20260811 14:36:01.425209 16315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:36:01.432551 16315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:36:01.434541 16315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:36:01.434725 16315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:36:01.434957 16315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:36:01.435266 16315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:36:01.435321 16315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:01.435364 16315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:36:01.435391 16315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:01.441273 16315 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:36539
I20260811 14:36:01.441352 16534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:36539 every 8 connection(s)
I20260811 14:36:01.441710 16315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:36:01.447113 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 16315
I20260811 14:36:01.447266 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:36:01.447458 2461 external_mini_cluster.cc:1428] Setting key dcc4c4f4431258e17e8fc95b2e0a1288
Aug 11 14:36:01 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458961, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:36:01.452740 16306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:01.442598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:57189 (local address 127.2.103.126:42517)
0811 14:36:01.442697 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:01.442699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:01.442967 (+ 268us) server_negotiation.cc:407] Connection header received
0811 14:36:01.443521 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:01.443523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:01.443543 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:01.443590 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:01.444589 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:01.445110 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:01.446120 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:01.446337 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:01.446415 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:01.449349 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:36:01.449351 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:36:01.449351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:36:01.449365 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:36:01.451030 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:36:01.451608 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:36:01.451609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:36:01.451610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:36:01.451674 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:36:01.451972 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:36:01.451973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:36:01.451973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:36:01.452129 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:36:01.452202 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:01.452486 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 14:36:01.452585 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 14:36:01.452822 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:01.442577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.65:57189)
0811 14:36:01.442830 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:01.442870 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:01.443398 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:01.443680 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:01.443684 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:01.444279 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:01.444486 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:01.444491 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:01.445210 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:01.445211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:01.446025 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:01.446028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:01.446425 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:01.446426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:01.446557 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:01.447246 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:01.447273 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:01.449239 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:36:01.451120 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:36:01.451128 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:36:01.451131 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:36:01.451518 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:36:01.451750 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:36:01.451751 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:36:01.451751 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:36:01.451898 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:36:01.452201 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:36:01.452206 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:36:01.452340 (+ 134us) client_negotiation.cc:769] Sending connection context
0811 14:36:01.452436 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 14:36:01.452577 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":154,"threads_started":1}
I20260811 14:36:01.453349 16537 heartbeater.cc:347] Connected to a master server at 127.2.103.126:42517
I20260811 14:36:01.453461 16537 heartbeater.cc:464] Registering TS with master...
I20260811 14:36:01.453691 16537 heartbeater.cc:511] Master 127.2.103.126:42517 requested a full tablet report, sending...
I20260811 14:36:01.454304 16219 ts_manager.cc:194] Registered new tserver with Master: 172eeef9ca314d87b8b8e1888ace2917 (127.2.103.65:36539)
I20260811 14:36:01.455035 16219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:57189
WARNING: no policy specified for kudu/127.2.103.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
I20260811 14:36:01.478489 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:42517
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:37253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:36:01.564777 16547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:36:01.565068 16547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:36:01.565110 16547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:36:01.566329 16547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:36:01.566406 16547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:36:01.566453 16547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:36:01.568136 16547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:37253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:42517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:36:01.568461 16547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:36:01.568727 16547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 14:36:01.574110 16547 server_base.cc:1034] running on GCE node
W20260811 14:36:01.574090 16557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:01.574076 16554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:01.574090 16553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:01.574587 16547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:36:01.574965 16547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:36:01.576138 16547 hybrid_clock.cc:648] HybridClock initialized: now 1786458961576118 us; error 32 us; skew 500 ppm
Aug 11 14:36:01 dist-test-slave-twwt krb5kdc[16136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458961, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:36:01.579212 16547 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:36:01.579739 16547 webserver.cc:460] Webserver started at http://127.2.103.66:37567/ using document root <none> and password file <none>
I20260811 14:36:01.580027 16547 fs_manager.cc:359] Metadata directory not provided
I20260811 14:36:01.580086 16547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:36:01.580299 16547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:36:01.581554 16547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "1b0f4963bb0144a4a8e6f99d80a8298b"
format_stamp: "Formatted at 2026-08-11 14:36:01 on dist-test-slave-twwt"
server_key: "5015829980c9c1d4e75f0781f980f950"
server_key_iv: "9e44c29a6f1d83eea36d2ca66fa7a849"
server_key_version: "encryptionkey@0"
I20260811 14:36:01.581983 16547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "1b0f4963bb0144a4a8e6f99d80a8298b"
format_stamp: "Formatted at 2026-08-11 14:36:01 on dist-test-slave-twwt"
server_key: "5015829980c9c1d4e75f0781f980f950"
server_key_iv: "9e44c29a6f1d83eea36d2ca66fa7a849"
server_key_version: "encryptionkey@0"
I20260811 14:36:01.583611 16547 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260811 14:36:01.584470 16568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:01.584676 16547 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:36:01.584784 16547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "1b0f4963bb0144a4a8e6f99d80a8298b"
format_stamp: "Formatted at 2026-08-11 14:36:01 on dist-test-slave-twwt"
server_key: "5015829980c9c1d4e75f0781f980f950"
server_key_iv: "9e44c29a6f1d83eea36d2ca66fa7a849"
server_key_version: "encryptionkey@0"
I20260811 14:36:01.584863 16547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:36:01.599701 16547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:36:01.602147 16547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:36:01.602332 16547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:36:01.602586 16547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:36:01.602998 16547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:36:01.603039 16547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:01.603061 16547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:36:01.603111 16547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:01.609315 16547 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:37091
I20260811 14:36:01.609356 16772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:37091 every 8 connection(s)
I20260811 14:36:01.609720 16547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:36:01.614578 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 16547
I20260811 14:36:01.614773 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:36:01.615015 2461 external_mini_cluster.cc:1428] Setting key 7a3fa8b3aae3ebfecd752dabd3aad37a
Aug 11 14:36:01 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458961, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:36:01.620556 16306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:01.610578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:44953 (local address 127.2.103.126:42517)
0811 14:36:01.610654 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:01.610656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:01.610881 (+ 225us) server_negotiation.cc:407] Connection header received
0811 14:36:01.611349 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:01.611351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:01.611364 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:01.611433 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:01.612478 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:01.613036 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:01.614055 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:01.614253 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:01.614317 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:01.617069 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:36:01.617071 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:36:01.617071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:36:01.617080 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:36:01.618779 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:36:01.619348 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:36:01.619349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:36:01.619349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:36:01.619406 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:36:01.619728 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:36:01.619729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:36:01.619729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:36:01.619897 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:36:01.619965 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:01.620376 (+ 411us) server_negotiation.cc:299] Negotiation successful
0811 14:36:01.620472 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 14:36:01.620905 16776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:01.610491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.66:44953)
0811 14:36:01.610694 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:01.610738 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:01.611245 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:01.611517 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:01.611521 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:01.612160 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:01.612368 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:01.612373 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:01.613130 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:01.613131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:01.613959 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:01.613961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:01.614356 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:01.614356 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:01.614468 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:01.615178 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:01.615189 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:01.616970 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:36:01.618898 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:36:01.618910 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:36:01.618914 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:36:01.619268 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:36:01.619471 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:36:01.619472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:36:01.619472 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:36:01.619634 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:36:01.619977 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:36:01.619984 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:36:01.620174 (+ 190us) client_negotiation.cc:769] Sending connection context
0811 14:36:01.620298 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 14:36:01.620577 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":101,"threads_started":1}
I20260811 14:36:01.621660 16773 heartbeater.cc:347] Connected to a master server at 127.2.103.126:42517
I20260811 14:36:01.621827 16773 heartbeater.cc:464] Registering TS with master...
I20260811 14:36:01.622076 16773 heartbeater.cc:511] Master 127.2.103.126:42517 requested a full tablet report, sending...
I20260811 14:36:01.622550 16219 ts_manager.cc:194] Registered new tserver with Master: 1b0f4963bb0144a4a8e6f99d80a8298b (127.2.103.66:37091)
I20260811 14:36:01.623152 16219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:44953
WARNING: no policy specified for kudu/127.2.103.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
I20260811 14:36:01.645641 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:42517
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:37253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:36:01.731838 16781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:36:01.732065 16781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:36:01.732090 16781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:36:01.733268 16781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:36:01.733309 16781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:36:01.733335 16781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:36:01.734788 16781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:37253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:42517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:36:01.734994 16781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:36:01.735188 16781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:36:01.740200 16792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:01.740209 16795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:01.740209 16789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:01.740670 16781 server_base.cc:1034] running on GCE node
I20260811 14:36:01.740940 16781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:36:01.741214 16781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:36:01.742396 16781 hybrid_clock.cc:648] HybridClock initialized: now 1786458961742379 us; error 28 us; skew 500 ppm
Aug 11 14:36:01 dist-test-slave-twwt krb5kdc[16136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458961, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:36:01.745214 16781 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:36:01.745708 16781 webserver.cc:460] Webserver started at http://127.2.103.67:46377/ using document root <none> and password file <none>
I20260811 14:36:01.745957 16781 fs_manager.cc:359] Metadata directory not provided
I20260811 14:36:01.746004 16781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:36:01.746163 16781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:36:01.747359 16781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "77c8c39a2b454e52856eaa1cf7e28aa8"
format_stamp: "Formatted at 2026-08-11 14:36:01 on dist-test-slave-twwt"
server_key: "c52c88cff665a2140c67a68933d36368"
server_key_iv: "6541baea793ee5fea8d355223f55bd55"
server_key_version: "encryptionkey@0"
I20260811 14:36:01.747766 16781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "77c8c39a2b454e52856eaa1cf7e28aa8"
format_stamp: "Formatted at 2026-08-11 14:36:01 on dist-test-slave-twwt"
server_key: "c52c88cff665a2140c67a68933d36368"
server_key_iv: "6541baea793ee5fea8d355223f55bd55"
server_key_version: "encryptionkey@0"
I20260811 14:36:01.749240 16781 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 14:36:01.750087 16805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:01.750362 16781 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:36:01.750473 16781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "77c8c39a2b454e52856eaa1cf7e28aa8"
format_stamp: "Formatted at 2026-08-11 14:36:01 on dist-test-slave-twwt"
server_key: "c52c88cff665a2140c67a68933d36368"
server_key_iv: "6541baea793ee5fea8d355223f55bd55"
server_key_version: "encryptionkey@0"
I20260811 14:36:01.750556 16781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:36:01.769362 16781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:36:01.772164 16781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:36:01.772361 16781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:36:01.772603 16781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:36:01.772984 16781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:36:01.773027 16781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:01.773052 16781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:36:01.773064 16781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:01.779249 16781 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:38519
I20260811 14:36:01.779335 16986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:38519 every 8 connection(s)
I20260811 14:36:01.779703 16781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:36:01.781771 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 16781
I20260811 14:36:01.781903 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:36:01.782100 2461 external_mini_cluster.cc:1428] Setting key ef06a2e5dc4f883e264d8ca319f94942
Aug 11 14:36:01 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458961, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:36:01.790268 16306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:01.780572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:42891 (local address 127.2.103.126:42517)
0811 14:36:01.780642 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:01.780644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:01.780823 (+ 179us) server_negotiation.cc:407] Connection header received
0811 14:36:01.781351 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:01.781352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:01.781365 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:01.781434 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:01.782584 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:01.783284 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:01.784266 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:01.784458 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:01.784523 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:01.786951 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:36:01.786953 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:36:01.786953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:36:01.786964 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:36:01.788689 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:36:01.789206 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:36:01.789207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:36:01.789208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:36:01.789279 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:36:01.789547 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:36:01.789548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:36:01.789549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:36:01.789681 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:36:01.789766 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:01.790061 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 14:36:01.790172 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 14:36:01.790468 16991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:01.780490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.67:42891)
0811 14:36:01.780661 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:01.780708 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:01.781222 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:01.781503 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:01.781507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:01.782186 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:01.782500 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:01.782507 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:01.783367 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:01.783368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:01.784172 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:01.784175 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:01.784539 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:01.784540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:01.784646 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:01.785289 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:01.785299 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:01.786837 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:36:01.788796 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:36:01.788805 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:36:01.788809 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:36:01.789135 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:36:01.789351 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:36:01.789353 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:36:01.789353 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:36:01.789478 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:36:01.789758 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:36:01.789766 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:36:01.789918 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 14:36:01.790025 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 14:36:01.790205 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
I20260811 14:36:01.791098 16988 heartbeater.cc:347] Connected to a master server at 127.2.103.126:42517
I20260811 14:36:01.791242 16988 heartbeater.cc:464] Registering TS with master...
I20260811 14:36:01.791453 16988 heartbeater.cc:511] Master 127.2.103.126:42517 requested a full tablet report, sending...
I20260811 14:36:01.791958 16219 ts_manager.cc:194] Registered new tserver with Master: 77c8c39a2b454e52856eaa1cf7e28aa8 (127.2.103.67:38519)
I20260811 14:36:01.792306 16219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:42891
I20260811 14:36:01.793926 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.65.keytab.
Entry for principal oryx/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.65.keytab.
WARNING: no policy specified for oryx/127.2.103.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.103.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.66.keytab.
Entry for principal oryx/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.66.keytab.
WARNING: no policy specified for oryx/127.2.103.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.103.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.67.keytab.
Entry for principal oryx/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.67.keytab.
I20260811 14:36:01.877388 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 16315
I20260811 14:36:01.902309 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 16547
I20260811 14:36:01.918408 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 16781
I20260811 14:36:01.932799 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 16166
I20260811 14:36:01.945878 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:42517
--webserver_interface=127.2.103.126
--webserver_port=39123
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:37253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:42517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:36:02.039325 17001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:36:02.039552 17001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:36:02.039577 17001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:36:02.040836 17001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:36:02.040887 17001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:36:02.040902 17001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:36:02.040915 17001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:36:02.040927 17001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:36:02.042491 17001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:37253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:42517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:42517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=39123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:36:02.042790 17001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:36:02.043095 17001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:36:02.048590 17007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:02.048590 17006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:02.048591 17009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:02.048784 17001 server_base.cc:1034] running on GCE node
I20260811 14:36:02.049217 17001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:36:02.049557 17001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:36:02.050690 17001 hybrid_clock.cc:648] HybridClock initialized: now 1786458962050656 us; error 60 us; skew 500 ppm
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458962, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:36:02.053452 17001 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:36:02.053947 17001 webserver.cc:460] Webserver started at http://127.2.103.126:39123/ using document root <none> and password file <none>
I20260811 14:36:02.054155 17001 fs_manager.cc:359] Metadata directory not provided
I20260811 14:36:02.054214 17001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:36:02.055650 17001 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:36:02.056388 17020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:02.056659 17001 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:36:02.056747 17001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "6ac44aec7c67497ba92abbc02ba87588"
format_stamp: "Formatted at 2026-08-11 14:36:01 on dist-test-slave-twwt"
server_key: "c012721ebd903b9821141d4b07323413"
server_key_iv: "9da005bd6ea53be0d0fd853c9fb58e21"
server_key_version: "encryptionkey@0"
I20260811 14:36:02.057108 17001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:36:02.070866 17001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:36:02.073015 17001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:36:02.073141 17001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:36:02.077647 17001 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:42517
I20260811 14:36:02.077684 17104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:42517 every 8 connection(s)
I20260811 14:36:02.078104 17001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:36:02.079733 17106 sys_catalog.cc:263] Verifying existing consensus state
I20260811 14:36:02.080453 17106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588: Bootstrap starting.
I20260811 14:36:02.081924 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 17001
I20260811 14:36:02.082262 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:36539
--local_ip_for_outbound_sockets=127.2.103.65
--tserver_master_addrs=127.2.103.126:42517
--webserver_port=35561
--webserver_interface=127.2.103.65
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:37253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.103.65
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:36:02.089605 17106 log.cc:824] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588: Log is configured to *not* fsync() on all Append() calls
I20260811 14:36:02.091565 17106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:36:02.091921 17106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588: Bootstrap complete.
I20260811 14:36:02.094308 17106 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac44aec7c67497ba92abbc02ba87588" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42517 } }
I20260811 14:36:02.094542 17106 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ac44aec7c67497ba92abbc02ba87588, State: Initialized, Role: FOLLOWER
I20260811 14:36:02.094740 17106 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac44aec7c67497ba92abbc02ba87588" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42517 } }
I20260811 14:36:02.094825 17106 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:36:02.094888 17106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:36:02.094936 17106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [term 1 FOLLOWER]: Advancing to term 2
I20260811 14:36:02.096101 17106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac44aec7c67497ba92abbc02ba87588" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42517 } }
I20260811 14:36:02.096221 17106 leader_election.cc:302] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ac44aec7c67497ba92abbc02ba87588; no voters:
I20260811 14:36:02.096439 17106 leader_election.cc:288] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 14:36:02.096509 17111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 14:36:02.096668 17111 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [term 2 LEADER]: Becoming Leader. State: Replica: 6ac44aec7c67497ba92abbc02ba87588, State: Running, Role: LEADER
I20260811 14:36:02.096735 17106 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:36:02.097609 17111 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac44aec7c67497ba92abbc02ba87588" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42517 } }
I20260811 14:36:02.097968 17114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ac44aec7c67497ba92abbc02ba87588. Latest consensus state: current_term: 2 leader_uuid: "6ac44aec7c67497ba92abbc02ba87588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac44aec7c67497ba92abbc02ba87588" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42517 } } }
I20260811 14:36:02.098060 17114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [sys.catalog]: This master's current role is: LEADER
I20260811 14:36:02.098157 17113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6ac44aec7c67497ba92abbc02ba87588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac44aec7c67497ba92abbc02ba87588" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42517 } } }
I20260811 14:36:02.098210 17113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588 [sys.catalog]: This master's current role is: LEADER
I20260811 14:36:02.098529 17125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:36:02.099056 17125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:36:02.099267 17125 catalog_manager.cc:1259] Loaded cluster ID: cdbd0420492843a8bfa81d75798cc8cc
I20260811 14:36:02.099459 17125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:36:02.100286 17125 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:36:02.100419 17125 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6ac44aec7c67497ba92abbc02ba87588: Loaded TSK: 0
I20260811 14:36:02.100559 17125 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 14:36:02.188333 17108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:36:02.188575 17108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:36:02.188602 17108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:36:02.189903 17108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:36:02.189960 17108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:36:02.189990 17108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:36:02.191713 17108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:37253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.65.keytab
--principal=oryx/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:36539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=35561
--tserver_master_addrs=127.2.103.126:42517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:36:02.192023 17108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:36:02.192304 17108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:36:02.198470 17146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:02.198669 17147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:02.198560 17108 server_base.cc:1034] running on GCE node
W20260811 14:36:02.198535 17149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:02.199146 17108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:36:02.199504 17108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:36:02.200840 17108 hybrid_clock.cc:648] HybridClock initialized: now 1786458962200697 us; error 46 us; skew 500 ppm
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458962, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:36:02.204600 17108 init.cc:377] Logged in from keytab as oryx/127.2.103.65@KRBTEST.COM (short username oryx)
I20260811 14:36:02.205219 17108 webserver.cc:460] Webserver started at http://127.2.103.65:35561/ using document root <none> and password file <none>
I20260811 14:36:02.205513 17108 fs_manager.cc:359] Metadata directory not provided
I20260811 14:36:02.205612 17108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:36:02.207322 17108 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:36:02.208202 17161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:02.208470 17108 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:36:02.208596 17108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "172eeef9ca314d87b8b8e1888ace2917"
format_stamp: "Formatted at 2026-08-11 14:36:01 on dist-test-slave-twwt"
server_key: "f6eeeede693872cb54a5e371042038a2"
server_key_iv: "6c700f6f88d856c2da4da96d9037fafc"
server_key_version: "encryptionkey@0"
I20260811 14:36:02.208987 17108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:36:02.232280 17108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:36:02.234901 17108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:36:02.235028 17108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:36:02.235350 17108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:36:02.235741 17108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:36:02.235781 17108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:02.235806 17108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:36:02.235845 17108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:02.242538 17108 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:36539
I20260811 14:36:02.242597 17342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:36539 every 8 connection(s)
I20260811 14:36:02.243006 17108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:36:02.249290 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 17108
I20260811 14:36:02.249662 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:37091
--local_ip_for_outbound_sockets=127.2.103.66
--tserver_master_addrs=127.2.103.126:42517
--webserver_port=37567
--webserver_interface=127.2.103.66
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:37253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.103.66
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:02.255703 17347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:02.243848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.65:54103)
0811 14:36:02.244052 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:02.244107 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:02.244750 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:02.247276 (+ 2526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:02.247280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:02.247973 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:02.248181 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:02.248187 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.249879 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.249880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:02.251202 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:02.251206 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.252100 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.252102 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:02.252278 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:02.252956 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:02.252965 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:02.255463 (+ 2498us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
W20260811 14:36:02.255754 17348 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:02.243910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:54103 (local address 127.2.103.126:42517)
0811 14:36:02.246759 (+ 2849us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:02.246770 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:02.246818 (+ 48us) server_negotiation.cc:407] Connection header received
0811 14:36:02.246967 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:02.246975 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:02.247035 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:02.247187 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:02.248316 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.249729 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.251314 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.251975 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.252114 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:02.255601 (+ 3487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.65:54103: BlockingRecv error: recv got EOF from 127.2.103.65:54103 (error 108)
Metrics: {"server-negotiator.queue_time_us":2788,"thread_start_us":117,"threads_started":1}
W20260811 14:36:02.256412 17345 heartbeater.cc:644] Failed to heartbeat to 127.2.103.126:42517 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.103.126:42517: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:02.261672 17347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:02.256684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.65:38637)
0811 14:36:02.256730 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:02.256762 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:02.256813 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:02.256970 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:02.256971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:02.257115 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:02.257162 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:02.257163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.257823 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.257824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:02.258332 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:02.258333 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.258772 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.258779 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:02.258855 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:02.259401 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:02.259406 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:02.261567 (+ 2161us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 14:36:02.261948 17348 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:02.256728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:38637 (local address 127.2.103.126:42517)
0811 14:36:02.256791 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:02.256793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:02.256803 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:36:02.256878 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:02.256878 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:02.256892 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:02.256966 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:02.257244 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.257733 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.258420 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.258681 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.258773 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:02.261855 (+ 3082us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.65:38637: BlockingRecv error: recv got EOF from 127.2.103.65:38637 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:02.267529 17347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:02.262059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.65:57179)
0811 14:36:02.262312 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:02.262322 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:02.262372 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:02.262580 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:02.262581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:02.262801 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:02.262841 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:02.262843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.263869 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.263870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:02.264412 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:02.264415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.264883 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.264884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:02.264927 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:02.265485 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:02.265489 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:02.267451 (+ 1962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236}
W20260811 14:36:02.267745 17348 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:02.262196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:57179 (local address 127.2.103.126:42517)
0811 14:36:02.262268 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:02.262271 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:02.262376 (+ 105us) server_negotiation.cc:407] Connection header received
0811 14:36:02.262426 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:02.262427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:02.262443 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:02.262515 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:02.262934 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.263752 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.264496 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.264808 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.264878 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:02.267671 (+ 2793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.65:57179: BlockingRecv error: recv got EOF from 127.2.103.65:57179 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260811 14:36:02.267815 17345 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 14:36:02.340055 17349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:36:02.340312 17349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:36:02.340350 17349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:36:02.341540 17349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:36:02.341614 17349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:36:02.341660 17349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:36:02.343230 17349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:37253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.66.keytab
--principal=oryx/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:37091
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=37567
--tserver_master_addrs=127.2.103.126:42517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:36:02.343524 17349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:36:02.343783 17349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:36:02.349026 17366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:02.349040 17361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:02.349089 17358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:02.349462 17349 server_base.cc:1034] running on GCE node
I20260811 14:36:02.349655 17349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:36:02.349887 17349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:36:02.351095 17349 hybrid_clock.cc:648] HybridClock initialized: now 1786458962351076 us; error 36 us; skew 500 ppm
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458962, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:36:02.353736 17349 init.cc:377] Logged in from keytab as oryx/127.2.103.66@KRBTEST.COM (short username oryx)
I20260811 14:36:02.354194 17349 webserver.cc:460] Webserver started at http://127.2.103.66:37567/ using document root <none> and password file <none>
I20260811 14:36:02.354417 17349 fs_manager.cc:359] Metadata directory not provided
I20260811 14:36:02.354473 17349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:36:02.355914 17349 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:36:02.356521 17378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:02.356701 17349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:36:02.356791 17349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "1b0f4963bb0144a4a8e6f99d80a8298b"
format_stamp: "Formatted at 2026-08-11 14:36:01 on dist-test-slave-twwt"
server_key: "5015829980c9c1d4e75f0781f980f950"
server_key_iv: "9e44c29a6f1d83eea36d2ca66fa7a849"
server_key_version: "encryptionkey@0"
I20260811 14:36:02.357239 17349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:36:02.371040 17349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:36:02.373481 17349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:36:02.373610 17349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:36:02.373826 17349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:36:02.374182 17349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:36:02.374218 17349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:02.374243 17349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:36:02.374255 17349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:02.380260 17349 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:37091
I20260811 14:36:02.380350 17565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:37091 every 8 connection(s)
I20260811 14:36:02.380722 17349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:02.387156 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 17349
I20260811 14:36:02.387492 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:38519
--local_ip_for_outbound_sockets=127.2.103.67
--tserver_master_addrs=127.2.103.126:42517
--webserver_port=46377
--webserver_interface=127.2.103.67
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:37253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.103.67
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 14:36:02.388943 17348 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:02.381538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:42147 (local address 127.2.103.126:42517)
0811 14:36:02.381588 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:02.381590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:02.381786 (+ 196us) server_negotiation.cc:407] Connection header received
0811 14:36:02.382295 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:02.382296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:02.382311 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:02.382375 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:02.383454 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.383979 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.385004 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.385211 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.385280 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:02.388884 (+ 3604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.66:42147: BlockingRecv error: recv got EOF from 127.2.103.66:42147 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:36:02.389024 17570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:02.381502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.66:42147)
0811 14:36:02.381668 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:02.381712 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:02.382206 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:02.382459 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:02.382462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:02.383135 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:02.383341 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:02.383347 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.384068 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.384069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:02.384881 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:02.384883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.385330 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.385331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:02.385449 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:02.386057 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:02.386066 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:02.388768 (+ 2702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
W20260811 14:36:02.389468 17566 heartbeater.cc:644] Failed to heartbeat to 127.2.103.126:42517 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.103.126:42517: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:02.394397 17570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:02.389738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.66:48767)
0811 14:36:02.389792 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:02.389801 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:02.389852 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:02.390010 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:02.390011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:02.390147 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:02.390194 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:02.390196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.390906 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.390907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:02.391411 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:02.391413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.391785 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.391786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:02.391861 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:02.392361 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:02.392366 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:02.394307 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 14:36:02.394454 17348 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:02.389805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:48767 (local address 127.2.103.126:42517)
0811 14:36:02.389871 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:02.389874 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:02.389882 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:36:02.389913 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:02.389914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:02.389931 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:02.390005 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:02.390272 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.390825 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.391506 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.391698 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.391786 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:02.394392 (+ 2606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.66:48767: BlockingRecv error: recv got EOF from 127.2.103.66:48767 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:02.399850 17570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:02.394756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.66:57673)
0811 14:36:02.394842 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:02.394853 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:02.394903 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:02.395029 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:02.395030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:02.395155 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:02.395184 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:02.395185 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.395810 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.395811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:02.396563 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:02.396565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.396916 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.396917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:02.396967 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:02.397469 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:02.397474 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:02.399710 (+ 2236us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 14:36:02.399883 17348 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:02.394798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:57673 (local address 127.2.103.126:42517)
0811 14:36:02.394848 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:02.394850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:02.394886 (+ 36us) server_negotiation.cc:407] Connection header received
0811 14:36:02.394964 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:02.394964 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:02.394973 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:02.395017 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:02.395268 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.395729 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.396654 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.396844 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.396926 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:02.399830 (+ 2904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.66:57673: BlockingRecv error: recv got EOF from 127.2.103.66:57673 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260811 14:36:02.400213 17566 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 14:36:02.474437 17571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:36:02.474658 17571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:36:02.474682 17571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:36:02.475914 17571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:36:02.475965 17571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:36:02.475992 17571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:36:02.477476 17571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:37253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/krb5kdc/oryx_127.2.103.67.keytab
--principal=oryx/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:38519
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=46377
--tserver_master_addrs=127.2.103.126:42517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:36:02.477733 17571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:36:02.477995 17571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:36:02.483518 17582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:02.483562 17571 server_base.cc:1034] running on GCE node
W20260811 14:36:02.483625 17583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:02.483518 17585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:02.484073 17571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:36:02.484308 17571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:36:02.485466 17571 hybrid_clock.cc:648] HybridClock initialized: now 1786458962485453 us; error 37 us; skew 500 ppm
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458962, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:36:02.488227 17571 init.cc:377] Logged in from keytab as oryx/127.2.103.67@KRBTEST.COM (short username oryx)
I20260811 14:36:02.488720 17571 webserver.cc:460] Webserver started at http://127.2.103.67:46377/ using document root <none> and password file <none>
I20260811 14:36:02.488950 17571 fs_manager.cc:359] Metadata directory not provided
I20260811 14:36:02.489006 17571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:36:02.490363 17571 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:36:02.491074 17598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:02.491241 17571 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:36:02.491338 17571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "77c8c39a2b454e52856eaa1cf7e28aa8"
format_stamp: "Formatted at 2026-08-11 14:36:01 on dist-test-slave-twwt"
server_key: "c52c88cff665a2140c67a68933d36368"
server_key_iv: "6541baea793ee5fea8d355223f55bd55"
server_key_version: "encryptionkey@0"
I20260811 14:36:02.491686 17571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:36:02.503515 17571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:36:02.505661 17571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:36:02.505792 17571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:36:02.506011 17571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:36:02.506287 17571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:36:02.506318 17571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:02.506341 17571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:36:02.506354 17571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:02.512331 17571 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:38519
I20260811 14:36:02.512463 17766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:38519 every 8 connection(s)
I20260811 14:36:02.512802 17571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:36:02.514209 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 17571
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 14:36:02.521176 17348 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:02.513624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:33501 (local address 127.2.103.126:42517)
0811 14:36:02.513712 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:02.513714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:02.513914 (+ 200us) server_negotiation.cc:407] Connection header received
0811 14:36:02.514505 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:02.514506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:02.514520 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:02.514572 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:02.515921 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.516438 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.517630 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.517935 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.518023 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:02.521114 (+ 3091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.67:33501: BlockingRecv error: recv got EOF from 127.2.103.67:33501 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 14:36:02.521261 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:02.513551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.67:33501)
0811 14:36:02.513782 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:02.513825 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:02.514391 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:02.514682 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:02.514685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:02.515541 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:02.515844 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:02.515850 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.516535 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.516536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:02.517536 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:02.517539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.518065 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.518066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:02.518198 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:02.518713 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:02.518722 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:02.521048 (+ 2326us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":106,"threads_started":1}
W20260811 14:36:02.521497 17767 heartbeater.cc:644] Failed to heartbeat to 127.2.103.126:42517 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.103.126:42517: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
I20260811 14:36:02.522817 17776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:02.514808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52778 (local address 127.2.103.126:42517)
0811 14:36:02.514945 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:02.514947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:02.514988 (+ 41us) server_negotiation.cc:407] Connection header received
0811 14:36:02.515011 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:02.515012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:02.515021 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:02.515071 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:02.515837 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.516341 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.517148 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.517461 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.517540 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:02.519191 (+ 1651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:36:02.519196 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:36:02.519196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:36:02.519216 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:36:02.521023 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:36:02.521639 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:36:02.521642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:36:02.521642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:36:02.521797 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:36:02.522008 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:36:02.522009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:36:02.522009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:36:02.522260 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:36:02.522412 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:02.522534 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 14:36:02.522685 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":94,"threads_started":1}
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:02.526078 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:02.521720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.67:57919)
0811 14:36:02.521771 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:02.521782 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:02.521823 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:02.521945 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:02.521946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:02.522075 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:02.522118 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:02.522120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.522832 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.522833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:02.523577 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:02.523580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.523978 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.523978 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:02.524031 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:02.524460 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:02.524467 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:02.526033 (+ 1566us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 14:36:02.526185 17348 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:02.521776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:57919 (local address 127.2.103.126:42517)
0811 14:36:02.521819 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:02.521822 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:02.521831 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:36:02.521885 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:02.521886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:02.521896 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:02.521944 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:02.522197 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.522751 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.523647 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.523921 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.523971 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:02.526118 (+ 2147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.67:57919: BlockingRecv error: recv got EOF from 127.2.103.67:57919 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:02 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:02.530470 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:02.526401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.67:56217)
0811 14:36:02.526455 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:02.526467 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:02.526508 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:02.526667 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:02.526668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:02.526789 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:02.526816 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:02.526817 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.527439 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.527440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:02.527929 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:02.527931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.528280 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.528280 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:02.528340 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:02.528689 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:02.528693 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:02.530393 (+ 1700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 14:36:02.530500 17348 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:02.526476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:56217 (local address 127.2.103.126:42517)
0811 14:36:02.526515 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:02.526517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:02.526541 (+ 24us) server_negotiation.cc:407] Connection header received
0811 14:36:02.526561 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:02.526562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:02.526572 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:02.526650 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:02.526932 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.527357 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.528013 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:02.528197 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:02.528267 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:02.530448 (+ 2181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.67:56217: BlockingRecv error: recv got EOF from 127.2.103.67:56217 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260811 14:36:02.530676 17767 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 14:36:03 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:03 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:03.273439 17780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:03.268326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.65:54109)
0811 14:36:03.268482 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:03.268498 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:03.268576 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:03.268809 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:03.268811 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:03.269044 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:03.269109 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:03.269112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:03.269865 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:03.269866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:03.270442 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:03.270444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:03.270795 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:03.270796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:03.270845 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:03.271392 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:03.271398 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:03.273361 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
W20260811 14:36:03.273579 17782 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:03.268428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:54109 (local address 127.2.103.126:42517)
0811 14:36:03.268591 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:03.268593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:03.268656 (+ 63us) server_negotiation.cc:407] Connection header received
0811 14:36:03.268690 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:03.268691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:03.268707 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:03.268774 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:03.269215 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:03.269771 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:03.270521 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:03.270728 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:03.270779 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:03.273464 (+ 2685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.65:54109: BlockingRecv error: recv got EOF from 127.2.103.65:54109 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
Aug 11 14:36:03 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:03 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:03.405584 17783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:03.400660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.66:52387)
0811 14:36:03.400847 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:03.400859 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:03.400954 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:03.401112 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:03.401113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:03.401312 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:03.401363 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:03.401365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:03.402051 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:03.402052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:03.402645 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:03.402648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:03.403080 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:03.403082 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:03.403142 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:03.403599 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:03.403604 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:03.405513 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":68,"threads_started":1}
W20260811 14:36:03.405689 17782 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:03.400748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:52387 (local address 127.2.103.126:42517)
0811 14:36:03.400840 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:03.400843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:03.400935 (+ 92us) server_negotiation.cc:407] Connection header received
0811 14:36:03.401014 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:03.401015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:03.401037 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:03.401103 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:03.401454 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:03.401961 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:03.402808 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:03.403000 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:03.403082 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:03.405620 (+ 2538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.66:52387: BlockingRecv error: recv got EOF from 127.2.103.66:52387 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 14:36:03 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:03 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:03.536204 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:03.531118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.67:43685)
0811 14:36:03.531305 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:03.531317 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:03.531423 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:03.531586 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:03.531588 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:03.531781 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:03.531831 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:03.531834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:03.532519 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:03.532520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:03.533104 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:03.533106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:03.533476 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:03.533477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:03.533537 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:03.534053 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:03.534066 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:03.536090 (+ 2024us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":100,"threads_started":1}
W20260811 14:36:03.536252 17782 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:03.531219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:43685 (local address 127.2.103.126:42517)
0811 14:36:03.531297 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:03.531299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:03.531402 (+ 103us) server_negotiation.cc:407] Connection header received
0811 14:36:03.531480 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:03.531481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:03.531498 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:03.531561 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:03.531944 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:03.532435 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:03.533203 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:03.533400 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:03.533478 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:03.536182 (+ 2704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.67:43685: BlockingRecv error: recv got EOF from 127.2.103.67:43685 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 11 14:36:04 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:04 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:04.281756 17787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:04.276058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.65:34413)
0811 14:36:04.276181 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:04.276193 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:04.276262 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:04.276449 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:04.276451 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:04.276640 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:04.276694 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:04.276696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:04.277868 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:04.277869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:04.278779 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:04.278783 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:04.279242 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:04.279243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:04.279311 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:04.279777 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:04.279782 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:04.281692 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":64,"threads_started":1}
W20260811 14:36:04.281932 17789 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:04.276136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:34413 (local address 127.2.103.126:42517)
0811 14:36:04.276271 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:04.276273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:04.276285 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:36:04.276319 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:04.276321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:04.276335 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:04.276435 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:04.276834 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:04.277749 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:04.278875 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:04.279137 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:04.279226 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:04.281825 (+ 2599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.65:34413: BlockingRecv error: recv got EOF from 127.2.103.65:34413 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
Aug 11 14:36:04 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:04 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:04.411578 17791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:04.406171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.66:38423)
0811 14:36:04.406366 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:04.406380 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:04.406483 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:04.406707 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:04.406708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:04.406899 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:04.406965 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:04.406966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:04.407673 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:04.407675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:04.408256 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:04.408258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:04.408653 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:04.408654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:04.408715 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:04.409358 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:04.409364 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:04.411479 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":100,"threads_started":1}
W20260811 14:36:04.411643 17789 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:04.406291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:38423 (local address 127.2.103.126:42517)
0811 14:36:04.406371 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:04.406374 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:04.406461 (+ 87us) server_negotiation.cc:407] Connection header received
0811 14:36:04.406587 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:04.406588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:04.406624 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:04.406696 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:04.407061 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:04.407585 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:04.408346 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:04.408566 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:04.408643 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:04.411576 (+ 2933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.66:38423: BlockingRecv error: recv got EOF from 127.2.103.66:38423 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 14:36:04 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:04 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:04.541718 17792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:04.536778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.67:38787)
0811 14:36:04.537018 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:04.537031 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:04.537245 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:04.537397 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:04.537398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:04.537575 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:04.537627 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:04.537628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:04.538253 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:04.538254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:04.538818 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:04.538819 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:04.539293 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:04.539294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:04.539375 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:04.539876 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:04.539881 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:04.541650 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":138,"threads_started":1}
W20260811 14:36:04.541833 17789 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:04.536971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:38787 (local address 127.2.103.126:42517)
0811 14:36:04.537048 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:04.537050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:04.537242 (+ 192us) server_negotiation.cc:407] Connection header received
0811 14:36:04.537311 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:04.537312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:04.537325 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:04.537390 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:04.537693 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:04.538179 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:04.538918 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:04.539201 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:04.539275 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:04.541748 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.67:38787: BlockingRecv error: recv got EOF from 127.2.103.67:38787 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 14:36:05 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:05 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:05.287169 17793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:05.282242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.65:50221)
0811 14:36:05.282387 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:05.282406 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:05.282459 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:05.282706 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:05.282707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:05.282904 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:05.282957 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:05.282958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:05.283699 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:05.283700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:05.284271 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:05.284274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:05.284651 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:05.284651 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:05.284695 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:05.285194 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:05.285199 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:05.287107 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":84,"threads_started":1}
W20260811 14:36:05.287353 17794 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:05.282375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:50221 (local address 127.2.103.126:42517)
0811 14:36:05.282514 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:05.282516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:05.282533 (+ 17us) server_negotiation.cc:407] Connection header received
0811 14:36:05.282568 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:05.282568 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:05.282579 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:05.282692 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:05.283041 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:05.283599 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:05.284368 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:05.284574 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:05.284634 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:05.287262 (+ 2628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.65:50221: BlockingRecv error: recv got EOF from 127.2.103.65:50221 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
Aug 11 14:36:05 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:05 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:05.417098 17797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:05.412113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.66:52325)
0811 14:36:05.412287 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:05.412300 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:05.412365 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:05.412526 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:05.412527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:05.412825 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:05.412896 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:05.412898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:05.413624 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:05.413625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:05.414172 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:05.414174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:05.414547 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:05.414548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:05.414592 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:05.415120 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:05.415125 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:05.417038 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":65,"threads_started":1}
W20260811 14:36:05.417277 17794 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:05.412276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:52325 (local address 127.2.103.126:42517)
0811 14:36:05.412342 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:05.412344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:05.412354 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:36:05.412418 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:05.412418 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:05.412429 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:05.412481 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:05.413000 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:05.413525 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:05.414266 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:05.414472 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:05.414548 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:05.417167 (+ 2619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.66:52325: BlockingRecv error: recv got EOF from 127.2.103.66:52325 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 11 14:36:05 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:05 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:05.547282 17798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:05.542295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.67:41035)
0811 14:36:05.542536 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:05.542546 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:05.542644 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:05.542877 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:05.542878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:05.543069 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:05.543119 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:05.543121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:05.543800 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:05.543802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:05.544335 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:05.544336 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:05.544721 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:05.544722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:05.544787 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:05.545295 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:05.545300 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:05.547174 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":119,"threads_started":1}
W20260811 14:36:05.547322 17794 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:05.542391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:41035 (local address 127.2.103.126:42517)
0811 14:36:05.542455 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:05.542457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:05.542632 (+ 175us) server_negotiation.cc:407] Connection header received
0811 14:36:05.542774 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:05.542775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:05.542791 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:05.542865 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:05.543204 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:05.543709 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:05.544423 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:05.544635 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:05.544726 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:05.547259 (+ 2533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.67:41035: BlockingRecv error: recv got EOF from 127.2.103.67:41035 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 11 14:36:06 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:06 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:06.292769 17800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:06.287701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.65:45705)
0811 14:36:06.287892 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:06.287904 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:06.287964 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:06.288122 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:06.288123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:06.288296 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:06.288347 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:06.288349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:06.289344 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:06.289345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:06.289872 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:06.289874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:06.290204 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:06.290204 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:06.290249 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:06.290777 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:06.290783 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:06.292695 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":128,"threads_started":1}
W20260811 14:36:06.292919 17801 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:06.287802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:45705 (local address 127.2.103.126:42517)
0811 14:36:06.287940 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:06.287942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:06.287961 (+ 19us) server_negotiation.cc:407] Connection header received
0811 14:36:06.288034 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:06.288034 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:06.288046 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:06.288106 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:06.288448 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:06.289255 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:06.289948 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:06.290136 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:06.290189 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:06.292798 (+ 2609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.65:45705: BlockingRecv error: recv got EOF from 127.2.103.65:45705 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":56,"threads_started":1}
Aug 11 14:36:06 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:06 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:06.427208 17802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:06.417719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.66:36259)
0811 14:36:06.417886 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:06.417898 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:06.417963 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:06.418103 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:06.418104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:06.418280 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:06.418332 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:06.418334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:06.419000 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:06.419001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:06.419531 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:06.419533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:06.419898 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:06.419899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:06.419945 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:06.425088 (+ 5143us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:06.425094 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:06.427141 (+ 2047us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":74,"threads_started":1}
W20260811 14:36:06.427372 17801 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:06.417813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:36259 (local address 127.2.103.126:42517)
0811 14:36:06.417895 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:06.417897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:06.417963 (+ 66us) server_negotiation.cc:407] Connection header received
0811 14:36:06.418023 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:06.418024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:06.418036 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:06.418078 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:06.418397 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:06.418922 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:06.419636 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:06.419828 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:06.419882 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:06.427286 (+ 7404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.66:36259: BlockingRecv error: recv got EOF from 127.2.103.66:36259 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 14:36:06 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:06 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:06.552821 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:06.547838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.67:33471)
0811 14:36:06.548004 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:06.548016 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:06.548106 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:06.548236 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:06.548237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:06.548440 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:06.548491 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:06.548493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:06.549173 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:06.549174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:06.549709 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:06.549719 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:06.550120 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:06.550121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:06.550181 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:06.550710 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:06.550715 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:06.552762 (+ 2047us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":103,"threads_started":1}
W20260811 14:36:06.552997 17801 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:06.548040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:33471 (local address 127.2.103.126:42517)
0811 14:36:06.548110 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:06.548112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:06.548133 (+ 21us) server_negotiation.cc:407] Connection header received
0811 14:36:06.548161 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:06.548162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:06.548176 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:06.548233 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:06.548580 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:06.549084 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:06.549819 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:06.550044 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:06.550121 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:06.552914 (+ 2793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.67:33471: BlockingRecv error: recv got EOF from 127.2.103.67:33471 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 14:36:07 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:07 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:07.298166 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:07.293244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.65:44243)
0811 14:36:07.293401 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:07.293413 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:07.293468 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:07.293626 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:07.293626 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:07.293793 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:07.293849 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:07.293851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:07.294549 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:07.294550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:07.295244 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:07.295247 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:07.295592 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:07.295593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:07.295638 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:07.296066 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:07.296070 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:07.298104 (+ 2034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":97,"threads_started":1}
W20260811 14:36:07.298281 17810 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:07.293347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:44243 (local address 127.2.103.126:42517)
0811 14:36:07.293508 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:07.293510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:07.293519 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:36:07.293542 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:07.293542 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:07.293552 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:07.293611 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:07.293951 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:07.294477 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:07.295330 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:07.295528 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:07.295580 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:07.298202 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.65:44243: BlockingRecv error: recv got EOF from 127.2.103.65:44243 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":105,"threads_started":1}
Aug 11 14:36:07 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:07 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:07.432605 17811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:07.427764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.66:44345)
0811 14:36:07.427940 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:07.427954 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:07.428034 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:07.428173 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:07.428175 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:07.428357 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:07.428410 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:07.428412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:07.429092 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:07.429092 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:07.429613 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:07.429614 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:07.430069 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:07.430070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:07.430146 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:07.430704 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:07.430710 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:07.432544 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":79,"threads_started":1}
W20260811 14:36:07.432778 17810 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:07.427959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:44345 (local address 127.2.103.126:42517)
0811 14:36:07.428037 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:07.428039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:07.428068 (+ 29us) server_negotiation.cc:407] Connection header received
0811 14:36:07.428095 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:07.428096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:07.428107 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:07.428148 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:07.428497 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:07.429021 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:07.429697 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:07.429973 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:07.430066 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:07.432688 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.66:44345: BlockingRecv error: recv got EOF from 127.2.103.66:44345 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 14:36:07 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:07 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:07.558182 17812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:07.553388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.67:49481)
0811 14:36:07.553549 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:07.553559 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:07.553622 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:07.553772 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:07.553773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:07.553943 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:07.553995 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:07.553996 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:07.554675 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:07.554676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:07.555242 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:07.555244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:07.555572 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:07.555573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:07.555612 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:07.556062 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:07.556067 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:07.558117 (+ 2050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
W20260811 14:36:07.558266 17810 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:07.553480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:49481 (local address 127.2.103.126:42517)
0811 14:36:07.553561 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:07.553563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:07.553624 (+ 61us) server_negotiation.cc:407] Connection header received
0811 14:36:07.553685 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:07.553686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:07.553698 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:07.553742 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:07.554060 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:07.554550 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:07.555321 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:07.555506 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:07.555558 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:07.558216 (+ 2658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.67:49481: BlockingRecv error: recv got EOF from 127.2.103.67:49481 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 14:36:08 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:08 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:08.303447 17813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:08.298781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.65:53927)
0811 14:36:08.298972 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:08.298985 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:08.299058 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:08.299254 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:08.299255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:08.299465 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:08.299518 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:08.299521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:08.300164 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:08.300165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:08.300709 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:08.300710 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:08.301047 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:08.301048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:08.301091 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:08.301527 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:08.301531 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:08.303386 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":85,"threads_started":1}
W20260811 14:36:08.303589 17814 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:08.298852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:53927 (local address 127.2.103.126:42517)
0811 14:36:08.298981 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:08.298984 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:08.299056 (+ 72us) server_negotiation.cc:407] Connection header received
0811 14:36:08.299149 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:08.299151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:08.299166 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:08.299239 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:08.299594 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:08.300094 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:08.300785 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:08.300982 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:08.301031 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:08.303496 (+ 2465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.65:53927: BlockingRecv error: recv got EOF from 127.2.103.65:53927 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":53,"threads_started":1}
Aug 11 14:36:08 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:08 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:08.438016 17815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:08.433102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.66:55575)
0811 14:36:08.433304 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:08.433316 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:08.433393 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:08.433578 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:08.433580 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:08.433778 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:08.433826 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:08.433828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:08.434513 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:08.434515 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:08.435100 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:08.435102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:08.435483 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:08.435484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:08.435545 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:08.436053 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:08.436059 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:08.437954 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":112,"threads_started":1}
W20260811 14:36:08.438143 17814 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:08.433214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:55575 (local address 127.2.103.126:42517)
0811 14:36:08.433291 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:08.433294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:08.433386 (+ 92us) server_negotiation.cc:407] Connection header received
0811 14:36:08.433480 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:08.433481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:08.433494 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:08.433570 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:08.433910 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:08.434416 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:08.435195 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:08.435392 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:08.435470 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:08.438064 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.66:55575: BlockingRecv error: recv got EOF from 127.2.103.66:55575 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 11 14:36:08 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:08 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:08.563771 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:08.558836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.67:41313)
0811 14:36:08.558958 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:08.558977 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:08.559044 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:08.559184 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:08.559185 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:08.559433 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:08.559499 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:08.559502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:08.560322 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:08.560324 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:08.560958 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:08.560962 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:08.561321 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:08.561322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:08.561367 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:08.561812 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:08.561818 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:08.563710 (+ 1892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
W20260811 14:36:08.563908 17814 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:08.558905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:41313 (local address 127.2.103.126:42517)
0811 14:36:08.558997 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:08.558999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:08.559026 (+ 27us) server_negotiation.cc:407] Connection header received
0811 14:36:08.559105 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:08.559106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:08.559119 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:08.559171 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:08.559577 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:08.560227 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:08.561047 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:08.561244 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:08.561338 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:08.563818 (+ 2480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.67:41313: BlockingRecv error: recv got EOF from 127.2.103.67:41313 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 14:36:09 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:09 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:09.309152 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:09.304084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.65:50239)
0811 14:36:09.304249 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:09.304261 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:09.304323 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:09.304541 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:09.304543 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:09.304765 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:09.304831 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:09.304834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:09.305587 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:09.305588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:09.306329 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:09.306333 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:09.306809 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:09.306810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:09.306855 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:09.307293 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:09.307298 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:09.309091 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
W20260811 14:36:09.309279 17818 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:09.304158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:50239 (local address 127.2.103.126:42517)
0811 14:36:09.304293 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:09.304295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:09.304312 (+ 17us) server_negotiation.cc:407] Connection header received
0811 14:36:09.304406 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:09.304408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:09.304426 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:09.304532 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:09.304919 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:09.305491 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:09.306442 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:09.306736 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:09.306800 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:09.309194 (+ 2394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.65:50239: BlockingRecv error: recv got EOF from 127.2.103.65:50239 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
Aug 11 14:36:09 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:09 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:09.443516 17821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:09.438669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.66:60949)
0811 14:36:09.438837 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:09.438853 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:09.438922 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:09.439110 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:09.439112 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:09.439291 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:09.439344 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:09.439346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:09.440014 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:09.440015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:09.440560 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:09.440562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:09.440943 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:09.440944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:09.441011 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:09.441523 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:09.441536 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:09.443416 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
W20260811 14:36:09.443567 17818 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:09.438726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:60949 (local address 127.2.103.126:42517)
0811 14:36:09.438811 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:09.438814 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:09.438923 (+ 109us) server_negotiation.cc:407] Connection header received
0811 14:36:09.438988 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:09.438989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:09.439001 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:09.439044 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:09.439419 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:09.439917 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:09.440655 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:09.440862 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:09.440943 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:09.443502 (+ 2559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.66:60949: BlockingRecv error: recv got EOF from 127.2.103.66:60949 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 11 14:36:09 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:09 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:09.568962 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:09.564289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.67:45563)
0811 14:36:09.564503 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:09.564513 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:09.564572 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:09.564694 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:09.564695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:09.564893 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:09.564945 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:09.564947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:09.565599 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:09.565600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:09.566136 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:09.566138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:09.566453 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:09.566454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:09.566495 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:09.567055 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:09.567060 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:09.568900 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":134,"threads_started":1}
W20260811 14:36:09.569159 17818 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:09.564402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:45563 (local address 127.2.103.126:42517)
0811 14:36:09.564476 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:09.564479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:09.564574 (+ 95us) server_negotiation.cc:407] Connection header received
0811 14:36:09.564619 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:09.564620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:09.564634 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:09.564682 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:09.565011 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:09.565525 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:09.566211 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:09.566387 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:09.566440 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:09.569058 (+ 2618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.67:45563: BlockingRecv error: recv got EOF from 127.2.103.67:45563 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 14:36:10 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:10 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:10.314733 17826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:10.309704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.65:51603)
0811 14:36:10.309898 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:10.309918 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:10.309972 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:10.310146 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:10.310147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:10.310327 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:10.310379 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:10.310381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:10.311049 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:10.311050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:10.311584 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:10.311585 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:10.312007 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:10.312008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:10.312062 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:10.312618 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:10.312624 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:10.314597 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":154,"threads_started":1}
W20260811 14:36:10.314771 17827 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:10.309872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:51603 (local address 127.2.103.126:42517)
0811 14:36:10.310012 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:10.310014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:10.310022 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:36:10.310046 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:10.310047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:10.310057 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:10.310123 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:10.310451 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:10.310977 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:10.311659 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:10.311937 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:10.311989 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:10.314716 (+ 2727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.65:51603: BlockingRecv error: recv got EOF from 127.2.103.65:51603 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
Aug 11 14:36:10 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:10 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:10.449162 17828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:10.444104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.66:58015)
0811 14:36:10.444291 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:10.444312 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:10.444379 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:10.444535 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:10.444536 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:10.444724 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:10.444787 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:10.444789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:10.445603 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:10.445604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:10.446214 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:10.446215 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:10.446564 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:10.446565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:10.446638 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:10.447102 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:10.447107 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:10.449096 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":109,"threads_started":1}
W20260811 14:36:10.449311 17827 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:10.444207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:58015 (local address 127.2.103.126:42517)
0811 14:36:10.444278 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:10.444280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:10.444378 (+ 98us) server_negotiation.cc:407] Connection header received
0811 14:36:10.444434 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:10.444434 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:10.444446 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:10.444506 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:10.444857 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:10.445516 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:10.446308 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:10.446492 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:10.446544 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:10.449227 (+ 2683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.66:58015: BlockingRecv error: recv got EOF from 127.2.103.66:58015 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 11 14:36:10 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:10 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:10.574523 17830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:10.569551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.67:38379)
0811 14:36:10.569718 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:10.569733 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:10.569835 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:10.570041 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:10.570042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:10.570319 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:10.570382 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:10.570384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:10.571088 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:10.571090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:10.571614 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:10.571615 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:10.571997 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:10.571998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:10.572043 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:10.572572 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:10.572578 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:10.574454 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":93,"threads_started":1}
W20260811 14:36:10.574641 17827 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:10.569667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:38379 (local address 127.2.103.126:42517)
0811 14:36:10.569731 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:10.569733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:10.569826 (+ 93us) server_negotiation.cc:407] Connection header received
0811 14:36:10.569933 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:10.569934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:10.569952 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:10.570014 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:10.570470 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:10.571009 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:10.571707 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:10.571917 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:10.571978 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:10.574552 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.67:38379: BlockingRecv error: recv got EOF from 127.2.103.67:38379 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 11 14:36:11 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:11 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:11.320104 17831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:11.315228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.65:35013)
0811 14:36:11.315384 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:11.315395 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:11.315448 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:11.315800 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:11.315801 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:11.315995 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:11.316047 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:11.316049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:11.316748 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:11.316749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:11.317266 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:11.317267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:11.317649 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:11.317650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:11.317701 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:11.318165 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:11.318169 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:11.320043 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
W20260811 14:36:11.320245 17836 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:11.315422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:35013 (local address 127.2.103.126:42517)
0811 14:36:11.315620 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:11.315623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:11.315631 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:36:11.315691 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:11.315692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:11.315704 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:11.315782 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:11.316158 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:11.316673 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:11.317358 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:11.317573 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:11.317633 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:11.320148 (+ 2515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.65:35013: BlockingRecv error: recv got EOF from 127.2.103.65:35013 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":76,"threads_started":1}
Aug 11 14:36:11 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:11 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:11.459121 17837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:11.449813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.66:48459)
0811 14:36:11.450028 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:11.450039 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:11.450104 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:11.450245 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:11.450246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:11.450437 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:11.450488 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:11.450490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:11.451190 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:11.451191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:11.451724 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:11.451725 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:11.452059 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:11.452060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:11.452100 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:11.456810 (+ 4710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:11.456817 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:11.459048 (+ 2231us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":129,"threads_started":1}
W20260811 14:36:11.459332 17836 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:11.449880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:48459 (local address 127.2.103.126:42517)
0811 14:36:11.449937 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:11.449940 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:11.450088 (+ 148us) server_negotiation.cc:407] Connection header received
0811 14:36:11.450165 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:11.450166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:11.450177 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:11.450219 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:11.450559 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:11.451116 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:11.451800 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:11.451992 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:11.452044 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:11.459257 (+ 7213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.66:48459: BlockingRecv error: recv got EOF from 127.2.103.66:48459 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 11 14:36:11 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:11 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:11.580320 17840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:11.575110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.67:49543)
0811 14:36:11.575287 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:11.575307 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:11.575399 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:11.575571 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:11.575572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:11.575809 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:11.575864 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:11.575866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:11.576579 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:11.576580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:11.577114 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:11.577115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:11.577485 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:11.577486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:11.577551 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:11.578108 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:11.578114 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:11.580224 (+ 2110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":123,"threads_started":1}
W20260811 14:36:11.580384 17836 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:11.575302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:49543 (local address 127.2.103.126:42517)
0811 14:36:11.575377 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:11.575379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:11.575392 (+ 13us) server_negotiation.cc:407] Connection header received
0811 14:36:11.575464 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:11.575465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:11.575481 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:11.575560 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:11.575962 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:11.576476 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:11.577208 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:11.577405 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:11.577484 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:11.580320 (+ 2836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.67:49543: BlockingRecv error: recv got EOF from 127.2.103.67:49543 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 14:36:12 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:12 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:12.325744 17841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:12.320648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.65:46393)
0811 14:36:12.320790 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:12.320803 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:12.320862 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:12.321180 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:12.321181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:12.321479 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:12.321561 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:12.321563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:12.322245 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:12.322246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:12.322890 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:12.322892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:12.323252 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:12.323253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:12.323294 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:12.323807 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:12.323812 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:12.325673 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
W20260811 14:36:12.325908 17842 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:12.320761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:46393 (local address 127.2.103.126:42517)
0811 14:36:12.320896 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:12.320898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:12.320908 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:36:12.320968 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:12.320969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:12.320979 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:12.321046 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:12.321643 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:12.322168 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:12.322967 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:12.323177 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:12.323245 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:12.325802 (+ 2557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.65:46393: BlockingRecv error: recv got EOF from 127.2.103.65:46393 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
Aug 11 14:36:12 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:12 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:12.465039 17843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:12.459622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.66:40633)
0811 14:36:12.459761 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:12.459780 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:12.459835 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:12.460063 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:12.460064 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:12.460246 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:12.460300 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:12.460301 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:12.461335 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:12.461336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:12.461857 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:12.461859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:12.462204 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:12.462205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:12.462246 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:12.462836 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:12.462842 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:12.464977 (+ 2135us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
W20260811 14:36:12.465219 17842 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:12.459796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:40633 (local address 127.2.103.126:42517)
0811 14:36:12.459885 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:12.459887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:12.459896 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:36:12.459949 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:12.459950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:12.459965 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:12.460042 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:12.460408 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:12.461250 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:12.461948 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:12.462136 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:12.462187 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:12.465124 (+ 2937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.66:40633: BlockingRecv error: recv got EOF from 127.2.103.66:40633 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 14:36:12 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:12 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:12.585829 17845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:12.580945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.67:45913)
0811 14:36:12.581117 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:12.581133 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:12.581236 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:12.581382 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:12.581382 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:12.581554 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:12.581606 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:12.581607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:12.582348 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:12.582349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:12.583031 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:12.583034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:12.583368 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:12.583369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:12.583413 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:12.583858 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:12.583862 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:12.585714 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":79,"threads_started":1}
W20260811 14:36:12.585866 17842 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:12.581112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:45913 (local address 127.2.103.126:42517)
0811 14:36:12.581168 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:12.581170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:12.581207 (+ 37us) server_negotiation.cc:407] Connection header received
0811 14:36:12.581286 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:12.581288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:12.581300 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:12.581350 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:12.581676 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:12.582253 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:12.583108 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:12.583295 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:12.583349 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:12.585799 (+ 2450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.67:45913: BlockingRecv error: recv got EOF from 127.2.103.67:45913 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 11 14:36:13 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:13 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:13.331401 17847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:13.326342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.65:53043)
0811 14:36:13.326555 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:13.326567 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:13.326682 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:13.326899 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:13.326900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:13.327093 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:13.327144 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:13.327145 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:13.327791 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:13.327792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:13.328341 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:13.328342 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:13.328710 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:13.328711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:13.328751 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:13.329236 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:13.329241 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:13.331282 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":103,"threads_started":1}
W20260811 14:36:13.331527 17849 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:13.326462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:53043 (local address 127.2.103.126:42517)
0811 14:36:13.326577 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:13.326579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:13.326650 (+ 71us) server_negotiation.cc:407] Connection header received
0811 14:36:13.326775 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:13.326776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:13.326787 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:13.326848 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:13.327214 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:13.327721 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:13.328440 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:13.328641 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:13.328698 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:13.331425 (+ 2727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.65:53043: BlockingRecv error: recv got EOF from 127.2.103.65:53043 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":54,"threads_started":1}
Aug 11 14:36:13 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:13 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:13.470252 17851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:13.465626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.66:43183)
0811 14:36:13.465884 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:13.465895 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:13.465954 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:13.466094 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:13.466095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:13.466276 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:13.466328 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:13.466329 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:13.466996 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:13.466997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:13.467537 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:13.467539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:13.467864 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:13.467865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:13.467904 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:13.468373 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:13.468378 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:13.470182 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":184,"threads_started":1}
W20260811 14:36:13.470355 17849 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:13.465686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:43183 (local address 127.2.103.126:42517)
0811 14:36:13.465755 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:13.465757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:13.465955 (+ 198us) server_negotiation.cc:407] Connection header received
0811 14:36:13.466003 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:13.466004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:13.466015 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:13.466067 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:13.466398 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:13.466926 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:13.467609 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:13.467792 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:13.467843 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:13.470287 (+ 2444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.66:43183: BlockingRecv error: recv got EOF from 127.2.103.66:43183 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 14:36:13 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:13 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:13.590989 17853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:13.586384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.67:56135)
0811 14:36:13.586554 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:13.586569 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:13.586642 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:13.586793 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:13.586794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:13.586981 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:13.587032 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:13.587034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:13.587665 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:13.587665 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:13.588213 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:13.588214 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:13.588539 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:13.588540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:13.588580 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:13.589038 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:13.589043 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:13.590905 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":111,"threads_started":1}
W20260811 14:36:13.591084 17849 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:13.586468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:56135 (local address 127.2.103.126:42517)
0811 14:36:13.586515 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:13.586517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:13.586624 (+ 107us) server_negotiation.cc:407] Connection header received
0811 14:36:13.586706 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:13.586707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:13.586719 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:13.586769 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:13.587105 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:13.587593 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:13.588287 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:13.588473 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:13.588525 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:13.591003 (+ 2478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.67:56135: BlockingRecv error: recv got EOF from 127.2.103.67:56135 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 11 14:36:14 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:14 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:14.336992 17855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:14.332050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.65:59967)
0811 14:36:14.332206 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:14.332217 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:14.332274 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:14.332429 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:14.332430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:14.332612 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:14.332659 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:14.332661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:14.333314 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:14.333314 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:14.333926 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:14.333929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:14.334346 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:14.334348 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:14.334403 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:14.334996 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:14.335001 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:14.336898 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
W20260811 14:36:14.337078 17856 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:14.332088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:59967 (local address 127.2.103.126:42517)
0811 14:36:14.332213 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:14.332215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:14.332276 (+ 61us) server_negotiation.cc:407] Connection header received
0811 14:36:14.332325 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:14.332326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:14.332338 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:14.332402 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:14.332730 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:14.333244 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:14.334027 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:14.334232 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:14.334309 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:14.337006 (+ 2697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.65:59967: BlockingRecv error: recv got EOF from 127.2.103.65:59967 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
Aug 11 14:36:14 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:14 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:14.475698 17857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:14.470867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.66:44453)
0811 14:36:14.471000 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:14.471012 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:14.471083 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:14.471232 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:14.471234 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:14.471414 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:14.471468 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:14.471469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:14.472132 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:14.472133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:14.472656 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:14.472657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:14.473048 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:14.473049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:14.473091 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:14.473564 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:14.473568 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:14.475637 (+ 2069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
W20260811 14:36:14.475832 17856 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:14.470954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:44453 (local address 127.2.103.126:42517)
0811 14:36:14.471030 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:14.471032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:14.471092 (+ 60us) server_negotiation.cc:407] Connection header received
0811 14:36:14.471129 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:14.471130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:14.471143 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:14.471210 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:14.471533 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:14.472051 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:14.472747 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:14.472974 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:14.473033 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:14.475749 (+ 2716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.66:44453: BlockingRecv error: recv got EOF from 127.2.103.66:44453 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 11 14:36:14 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:14 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:14.597041 17859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:14.591579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.67:55371)
0811 14:36:14.591795 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:14.591809 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:14.591938 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:14.592118 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:14.592120 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:14.592324 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:14.592372 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:14.592374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:14.593111 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:14.593112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:14.593645 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:14.593647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:14.594027 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:14.594028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:14.594090 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:14.594680 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:14.594685 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:14.596978 (+ 2293us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":116,"threads_started":1}
W20260811 14:36:14.597163 17856 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:14.591712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:55371 (local address 127.2.103.126:42517)
0811 14:36:14.591799 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:14.591802 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:14.591888 (+ 86us) server_negotiation.cc:407] Connection header received
0811 14:36:14.592023 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:14.592024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:14.592039 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:14.592111 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:14.592463 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:14.593021 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:14.593744 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:14.593941 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:14.594014 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:14.597112 (+ 3098us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.67:55371: BlockingRecv error: recv got EOF from 127.2.103.67:55371 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 14:36:15 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:15 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:15.343111 17860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:15.337600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.65:56569)
0811 14:36:15.337841 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:15.337853 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:15.337916 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:15.338127 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:15.338128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:15.338333 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:15.338387 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:15.338389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:15.339160 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:15.339162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:15.339743 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:15.339745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:15.340189 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:15.340190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:15.340240 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:15.340781 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:15.340786 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:15.342992 (+ 2206us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":113,"threads_started":1}
W20260811 14:36:15.343179 17861 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:15.337798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:56569 (local address 127.2.103.126:42517)
0811 14:36:15.337932 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:15.337934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:15.337945 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:36:15.337970 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:15.337971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:15.337983 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:15.338046 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:15.338510 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:15.339070 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:15.339854 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:15.340103 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:15.340177 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:15.343090 (+ 2913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.65:56569: BlockingRecv error: recv got EOF from 127.2.103.65:56569 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
Aug 11 14:36:15 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:15 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:15.481179 17862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:15.476277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.66:50719)
0811 14:36:15.476462 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:15.476495 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:15.476566 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:15.476759 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:15.476760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:15.476962 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:15.477010 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:15.477012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:15.477701 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:15.477702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:15.478242 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:15.478244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:15.478659 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:15.478661 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:15.478730 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:15.479201 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:15.479206 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:15.481079 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":96,"threads_started":1}
W20260811 14:36:15.481233 17861 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:15.476444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:50719 (local address 127.2.103.126:42517)
0811 14:36:15.476533 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:15.476535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:15.476548 (+ 13us) server_negotiation.cc:407] Connection header received
0811 14:36:15.476658 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:15.476659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:15.476673 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:15.476750 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:15.477100 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:15.477612 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:15.478337 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:15.478546 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:15.478655 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:15.481168 (+ 2513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.66:50719: BlockingRecv error: recv got EOF from 127.2.103.66:50719 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 14:36:15 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:15 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:15.603013 17864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:15.597808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.67:42731)
0811 14:36:15.598026 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:15.598039 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:15.598140 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:15.598310 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:15.598311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:15.598517 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:15.598565 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:15.598567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:15.599370 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:15.599372 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:15.600082 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:15.600084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:15.600460 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:15.600461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:15.600520 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:15.601039 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:15.601044 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:15.602940 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":160,"threads_started":1}
W20260811 14:36:15.603127 17861 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:15.597892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:42731 (local address 127.2.103.126:42517)
0811 14:36:15.597998 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:15.598001 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:15.598118 (+ 117us) server_negotiation.cc:407] Connection header received
0811 14:36:15.598204 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:15.598205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:15.598225 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:15.598301 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:15.598708 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:15.599273 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:15.600178 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:15.600376 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:15.600449 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:15.603040 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.67:42731: BlockingRecv error: recv got EOF from 127.2.103.67:42731 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 14:36:16 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:16 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:16.348634 17865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:16.343733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.65:47265)
0811 14:36:16.343885 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:16.343923 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:16.343981 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:16.344117 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:16.344118 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:16.344303 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:16.344355 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:16.344357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:16.345033 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:16.345034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:16.345566 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:16.345568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:16.345928 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:16.345928 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:16.345972 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:16.346489 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:16.346494 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:16.348531 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
W20260811 14:36:16.348701 17866 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:16.343785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:47265 (local address 127.2.103.126:42517)
0811 14:36:16.343963 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:16.343966 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:16.343976 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:36:16.344045 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:16.344045 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:16.344058 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:16.344110 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:16.344427 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:16.344948 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:16.345651 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:16.345841 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:16.345916 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:16.348630 (+ 2714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.65:47265: BlockingRecv error: recv got EOF from 127.2.103.65:47265 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":84,"threads_started":1}
Aug 11 14:36:16 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:16 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:16.492923 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:16.481731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.66:41981)
0811 14:36:16.481933 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:16.481948 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:16.482046 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:16.482208 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:16.482209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:16.482430 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:16.482480 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:16.482481 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:16.483240 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:16.483241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:16.483797 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:16.483799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:16.484174 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:16.484175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:16.484238 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:16.490839 (+ 6601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:16.490845 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:16.492814 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":102,"threads_started":1}
W20260811 14:36:16.493014 17866 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:16.481834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:41981 (local address 127.2.103.126:42517)
0811 14:36:16.481937 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:16.481940 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:16.482022 (+ 82us) server_negotiation.cc:407] Connection header received
0811 14:36:16.482109 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:16.482110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:16.482123 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:16.482200 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:16.482571 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:16.483141 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:16.483893 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:16.484094 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:16.484174 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:16.492952 (+ 8778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.66:41981: BlockingRecv error: recv got EOF from 127.2.103.66:41981 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 11 14:36:16 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:16 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.67@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:16.608973 17868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:16.603584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.67:43451)
0811 14:36:16.603820 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:16.603836 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:16.603939 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:16.604120 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:16.604121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:16.604325 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:16.604373 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:16.604375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:16.605084 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:16.605085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:16.605622 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:16.605624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:16.606021 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:16.606023 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:16.606090 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:16.606634 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:16.606641 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:16.608853 (+ 2212us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":124,"threads_started":1}
W20260811 14:36:16.609002 17866 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:16.603748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:43451 (local address 127.2.103.126:42517)
0811 14:36:16.603825 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:16.603827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:16.603917 (+ 90us) server_negotiation.cc:407] Connection header received
0811 14:36:16.604031 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:16.604032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:16.604048 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:16.604109 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:16.604475 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:16.604987 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:16.605713 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:16.605925 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:16.606015 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:16.608941 (+ 2926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.67:43451: BlockingRecv error: recv got EOF from 127.2.103.67:43451 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 11 14:36:17 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:17 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.65@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:17.354564 17870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:17.349375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.65:58781)
0811 14:36:17.349568 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:17.349585 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:17.349664 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:17.349888 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:17.349889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:17.350094 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:17.350178 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:17.350181 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:17.351110 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:17.351112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:17.351660 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:17.351662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:17.352060 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:17.352061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:17.352115 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:17.352636 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:17.352641 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:17.354502 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":87,"threads_started":1}
W20260811 14:36:17.354758 17871 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:17.349522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:58781 (local address 127.2.103.126:42517)
0811 14:36:17.349715 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:17.349718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:17.349732 (+ 14us) server_negotiation.cc:407] Connection header received
0811 14:36:17.349763 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:17.349764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:17.349784 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:17.349847 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:17.350278 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:17.351005 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:17.351776 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:17.351979 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:17.352045 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:17.354657 (+ 2612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.65:58781: BlockingRecv error: recv got EOF from 127.2.103.65:58781 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":99,"threads_started":1}
Aug 11 14:36:17 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:36:17 dist-test-slave-twwt krb5kdc[16136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.103.66@KRBTEST.COM for oryx/127.2.103.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:36:17.499405 17872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:17.493628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:42517 (local address 127.2.103.66:44561)
0811 14:36:17.493847 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:17.493862 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:17.493940 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:17.494136 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:17.494138 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:17.494405 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:17.494467 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:17.494469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:17.495292 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:17.495293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:17.496059 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:17.496062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:17.496463 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:17.496464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:17.496515 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:17.497061 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:17.497066 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:17.499304 (+ 2238us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.103.126:42517: Server oryx/127.2.103.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":100,"threads_started":1}
W20260811 14:36:17.499500 17871 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:17.493779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:44561 (local address 127.2.103.126:42517)
0811 14:36:17.493845 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:17.493848 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:17.493915 (+ 67us) server_negotiation.cc:407] Connection header received
0811 14:36:17.494025 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:17.494026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:17.494042 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:17.494109 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:17.494585 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:17.495194 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:17.496139 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:17.496368 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:17.496432 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:17.499431 (+ 2999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.103.66:44561: BlockingRecv error: recv got EOF from 127.2.103.66:44561 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 14:36:17.515784 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 17108
I20260811 14:36:17.533784 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 17349
I20260811 14:36:17.543956 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 17571
I20260811 14:36:17.557828 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 17001
2026-08-11T14:36:17Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17885 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T14:36:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:36:17Z Disabled control of system clock
I20260811 14:36:17.598668 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:37511
--webserver_interface=127.2.103.126
--webserver_port=0
--builtin_ntp_servers=127.2.103.84:38713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:37511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 14:36:17.684902 17876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:36:17.685119 17876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:36:17.685140 17876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:36:17.686290 17876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:36:17.686328 17876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:36:17.686340 17876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:36:17.686353 17876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:36:17.686365 17876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:36:17.687757 17876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:38713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:37511
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:37511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:36:17.687983 17876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:36:17.688256 17876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:36:17.693940 17890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:17.693950 17885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:17.694006 17876 server_base.cc:1034] running on GCE node
W20260811 14:36:17.694020 17886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:17.694579 17876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:36:17.695021 17876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:36:17.696206 17876 hybrid_clock.cc:648] HybridClock initialized: now 1786458977696192 us; error 47 us; skew 500 ppm
I20260811 14:36:17.697568 17876 webserver.cc:460] Webserver started at http://127.2.103.126:44851/ using document root <none> and password file <none>
I20260811 14:36:17.697870 17876 fs_manager.cc:359] Metadata directory not provided
I20260811 14:36:17.697979 17876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:36:17.698127 17876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:36:17.699340 17876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "59d789737c9843588335cbb6d484d1f6"
format_stamp: "Formatted at 2026-08-11 14:36:17 on dist-test-slave-twwt"
server_key: "0898fd0ae9a1b8626a036a5f925a3ddd"
server_key_iv: "d8de9bd10b7757fead4f08ca6d81cb41"
server_key_version: "encryptionkey@0"
I20260811 14:36:17.699770 17876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "59d789737c9843588335cbb6d484d1f6"
format_stamp: "Formatted at 2026-08-11 14:36:17 on dist-test-slave-twwt"
server_key: "0898fd0ae9a1b8626a036a5f925a3ddd"
server_key_iv: "d8de9bd10b7757fead4f08ca6d81cb41"
server_key_version: "encryptionkey@0"
I20260811 14:36:17.701164 17876 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:36:17.701932 17899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:17.702106 17876 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:36:17.702219 17876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "59d789737c9843588335cbb6d484d1f6"
format_stamp: "Formatted at 2026-08-11 14:36:17 on dist-test-slave-twwt"
server_key: "0898fd0ae9a1b8626a036a5f925a3ddd"
server_key_iv: "d8de9bd10b7757fead4f08ca6d81cb41"
server_key_version: "encryptionkey@0"
I20260811 14:36:17.702301 17876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:36:17.716059 17876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:36:17.716426 17876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:36:17.716595 17876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:36:17.721076 17876 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:37511
I20260811 14:36:17.721114 17991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:37511 every 8 connection(s)
I20260811 14:36:17.721494 17876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:36:17.722263 17992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:36:17.723886 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 17876
I20260811 14:36:17.723973 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:36:17.724195 2461 external_mini_cluster.cc:1428] Setting key 22b2d720c38b924840294075b87017f7
I20260811 14:36:17.725747 17992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6: Bootstrap starting.
I20260811 14:36:17.726553 17992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6: Neither blocks nor log segments found. Creating new log.
I20260811 14:36:17.726918 17992 log.cc:824] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6: Log is configured to *not* fsync() on all Append() calls
I20260811 14:36:17.727715 17992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6: No bootstrap required, opened a new log
I20260811 14:36:17.730067 17992 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59d789737c9843588335cbb6d484d1f6" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 37511 } }
I20260811 14:36:17.730340 17992 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:36:17.730413 17992 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59d789737c9843588335cbb6d484d1f6, State: Initialized, Role: FOLLOWER
I20260811 14:36:17.730577 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:17.724875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35292 (local address 127.2.103.126:37511)
0811 14:36:17.725105 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:17.725115 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:17.725150 (+ 35us) server_negotiation.cc:407] Connection header received
0811 14:36:17.725701 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:17.725721 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:17.725806 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:17.725961 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:17.726156 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:17.727505 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:17.728323 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:17.728955 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:17.729100 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:17.729792 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:36:17.729797 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:36:17.729798 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:36:17.730014 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:36:17.730060 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:36:17.730075 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:36:17.730149 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:17.730247 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 14:36:17.730336 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":91,"threads_started":1}
I20260811 14:36:17.730543 17992 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59d789737c9843588335cbb6d484d1f6" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 37511 } }
I20260811 14:36:17.730661 17992 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:36:17.730700 17992 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:36:17.730741 17992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:36:17.731413 17992 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59d789737c9843588335cbb6d484d1f6" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 37511 } }
I20260811 14:36:17.731504 17992 leader_election.cc:302] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59d789737c9843588335cbb6d484d1f6; no voters:
I20260811 14:36:17.731630 17992 leader_election.cc:288] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:36:17.731714 18002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:36:17.731915 18002 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6 [term 1 LEADER]: Becoming Leader. State: Replica: 59d789737c9843588335cbb6d484d1f6, State: Running, Role: LEADER
I20260811 14:36:17.732038 17992 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:36:17.732057 18002 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59d789737c9843588335cbb6d484d1f6" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 37511 } }
I20260811 14:36:17.732458 18003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59d789737c9843588335cbb6d484d1f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59d789737c9843588335cbb6d484d1f6" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 37511 } } }
I20260811 14:36:17.732471 18004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59d789737c9843588335cbb6d484d1f6. Latest consensus state: current_term: 1 leader_uuid: "59d789737c9843588335cbb6d484d1f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59d789737c9843588335cbb6d484d1f6" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 37511 } } }
I20260811 14:36:17.732612 18003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6 [sys.catalog]: This master's current role is: LEADER
I20260811 14:36:17.732729 18004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6 [sys.catalog]: This master's current role is: LEADER
I20260811 14:36:17.733029 18014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:36:17.733566 18014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:36:17.735257 18014 catalog_manager.cc:1347] Generated new cluster ID: de9ee9345eda46738c71600db7817835
I20260811 14:36:17.735320 18014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:36:17.754101 18014 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:36:17.754803 18014 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:36:17.761379 18014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59d789737c9843588335cbb6d484d1f6: Generated new TSK 0
I20260811 14:36:17.761641 18014 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:36:17.765430 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:37511
--builtin_ntp_servers=127.2.103.84:38713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 14:36:17.884896 18026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:36:17.885269 18026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:36:17.885313 18026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:36:17.887771 18026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:36:17.887871 18026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:36:17.887931 18026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:36:17.891019 18026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:38713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:37511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:36:17.891353 18026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:36:17.891764 18026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:36:17.898180 18037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:17.898422 18036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:17.905047 18026 server_base.cc:1034] running on GCE node
W20260811 14:36:17.906230 18039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:17.906797 18026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:36:17.907196 18026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:36:17.908385 18026 hybrid_clock.cc:648] HybridClock initialized: now 1786458977908372 us; error 37 us; skew 500 ppm
I20260811 14:36:17.910434 18026 webserver.cc:460] Webserver started at http://127.2.103.65:38079/ using document root <none> and password file <none>
I20260811 14:36:17.910854 18026 fs_manager.cc:359] Metadata directory not provided
I20260811 14:36:17.910946 18026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:36:17.911126 18026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:36:17.913130 18026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "3b2da739b2504a5286b8596e0d312ce5"
format_stamp: "Formatted at 2026-08-11 14:36:17 on dist-test-slave-twwt"
server_key: "cde6c22df42759dba09f62a71039138c"
server_key_iv: "adb4c66963f6cbd342d073aa5be50cb8"
server_key_version: "encryptionkey@0"
I20260811 14:36:17.913677 18026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "3b2da739b2504a5286b8596e0d312ce5"
format_stamp: "Formatted at 2026-08-11 14:36:17 on dist-test-slave-twwt"
server_key: "cde6c22df42759dba09f62a71039138c"
server_key_iv: "adb4c66963f6cbd342d073aa5be50cb8"
server_key_version: "encryptionkey@0"
I20260811 14:36:17.915889 18026 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.003s
I20260811 14:36:17.917073 18049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:17.917318 18026 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:36:17.917443 18026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "3b2da739b2504a5286b8596e0d312ce5"
format_stamp: "Formatted at 2026-08-11 14:36:17 on dist-test-slave-twwt"
server_key: "cde6c22df42759dba09f62a71039138c"
server_key_iv: "adb4c66963f6cbd342d073aa5be50cb8"
server_key_version: "encryptionkey@0"
I20260811 14:36:17.917548 18026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:36:17.948498 18026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:36:17.948837 18026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:36:17.948958 18026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:36:17.949182 18026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:36:17.949587 18026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:36:17.949663 18026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:17.949715 18026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:36:17.949762 18026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:17.958402 18026 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:33643
I20260811 14:36:17.958457 18229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:33643 every 8 connection(s)
I20260811 14:36:17.958937 18026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:36:17.964535 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 18026
I20260811 14:36:17.964668 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:36:17.964895 2461 external_mini_cluster.cc:1428] Setting key e7cce807de0d73f18ab5488d3a1339a6
I20260811 14:36:17.964968 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:17.959803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:34149 (local address 127.2.103.126:37511)
0811 14:36:17.959874 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:17.959876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:17.960148 (+ 272us) server_negotiation.cc:407] Connection header received
0811 14:36:17.960692 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:17.960693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:17.960703 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:17.960769 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:17.961500 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:17.962001 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:17.963249 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:17.963465 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:17.963536 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:17.964545 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:36:17.964546 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:36:17.964547 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:36:17.964552 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:36:17.964565 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:36:17.964569 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:36:17.964611 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:17.964837 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 14:36:17.964872 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 14:36:17.965505 18234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:17.959774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:37511 (local address 127.2.103.65:34149)
0811 14:36:17.960002 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:17.960049 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:17.960601 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:17.960832 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:17.960835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:17.960844 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:17.961409 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:17.961415 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:17.962094 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:17.962095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:17.963091 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:17.963094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:17.963569 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:17.963570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:17.963701 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:17.964336 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:36:17.964337 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:17.964381 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:36:17.964422 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:36:17.964637 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:36:17.964645 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:36:17.964649 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:36:17.965118 (+ 469us) client_negotiation.cc:240] Negotiation successful
0811 14:36:17.965274 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":112,"threads_started":1}
I20260811 14:36:17.966063 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:37511
--builtin_ntp_servers=127.2.103.84:38713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 14:36:17.966102 18231 heartbeater.cc:347] Connected to a master server at 127.2.103.126:37511
I20260811 14:36:17.966200 18231 heartbeater.cc:464] Registering TS with master...
I20260811 14:36:17.966420 18231 heartbeater.cc:511] Master 127.2.103.126:37511 requested a full tablet report, sending...
I20260811 14:36:17.966979 17927 ts_manager.cc:194] Registered new tserver with Master: 3b2da739b2504a5286b8596e0d312ce5 (127.2.103.65:33643)
I20260811 14:36:17.967633 17927 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.103.65:34149
W20260811 14:36:18.052567 18236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:36:18.052801 18236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:36:18.052824 18236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:36:18.054044 18236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:36:18.054090 18236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:36:18.054116 18236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:36:18.055540 18236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:38713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:37511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:36:18.055809 18236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:36:18.056133 18236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:36:18.061533 18245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:18.061525 18243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:18.061537 18248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:18.061806 18236 server_base.cc:1034] running on GCE node
I20260811 14:36:18.062048 18236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:36:18.062275 18236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:36:18.063422 18236 hybrid_clock.cc:648] HybridClock initialized: now 1786458978063383 us; error 58 us; skew 500 ppm
I20260811 14:36:18.064781 18236 webserver.cc:460] Webserver started at http://127.2.103.66:38579/ using document root <none> and password file <none>
I20260811 14:36:18.065043 18236 fs_manager.cc:359] Metadata directory not provided
I20260811 14:36:18.065116 18236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:36:18.065275 18236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:36:18.066484 18236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "63a849306a734caf8af03b2a4f50fd61"
format_stamp: "Formatted at 2026-08-11 14:36:18 on dist-test-slave-twwt"
server_key: "6aca2a59793631a10dd24b10f29cf99c"
server_key_iv: "47b659abed920a7a3d1bd9386cfafef6"
server_key_version: "encryptionkey@0"
I20260811 14:36:18.066947 18236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "63a849306a734caf8af03b2a4f50fd61"
format_stamp: "Formatted at 2026-08-11 14:36:18 on dist-test-slave-twwt"
server_key: "6aca2a59793631a10dd24b10f29cf99c"
server_key_iv: "47b659abed920a7a3d1bd9386cfafef6"
server_key_version: "encryptionkey@0"
I20260811 14:36:18.068413 18236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:36:18.069135 18261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:18.069309 18236 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:36:18.069425 18236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "63a849306a734caf8af03b2a4f50fd61"
format_stamp: "Formatted at 2026-08-11 14:36:18 on dist-test-slave-twwt"
server_key: "6aca2a59793631a10dd24b10f29cf99c"
server_key_iv: "47b659abed920a7a3d1bd9386cfafef6"
server_key_version: "encryptionkey@0"
I20260811 14:36:18.069504 18236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:36:18.092214 18236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:36:18.092585 18236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:36:18.092715 18236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:36:18.092933 18236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:36:18.093297 18236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:36:18.093333 18236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:18.093355 18236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:36:18.093405 18236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:18.099485 18236 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:37257
I20260811 14:36:18.099562 18454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:37257 every 8 connection(s)
I20260811 14:36:18.099947 18236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:36:18.102744 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 18236
I20260811 14:36:18.102849 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:36:18.103035 2461 external_mini_cluster.cc:1428] Setting key 40e00073531c1b8b27f8613ad8b6d3b6
I20260811 14:36:18.104161 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:37511
--builtin_ntp_servers=127.2.103.84:38713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 14:36:18.106256 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:18.100882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:40969 (local address 127.2.103.126:37511)
0811 14:36:18.100963 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:18.100965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:18.101221 (+ 256us) server_negotiation.cc:407] Connection header received
0811 14:36:18.101720 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:18.101721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:18.101734 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:18.101806 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:18.102526 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:18.103449 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:18.104586 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:18.104848 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:18.104949 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:18.105940 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:36:18.105941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:36:18.105942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:36:18.105946 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:36:18.105960 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:36:18.105965 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:36:18.106010 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:18.106149 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 14:36:18.106177 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 14:36:18.106493 18457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:18.100834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:37511 (local address 127.2.103.66:40969)
0811 14:36:18.101064 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:18.101108 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:18.101615 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:18.101890 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:18.101895 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:18.101904 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:18.102430 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:18.102435 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:18.103535 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:18.103536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:18.104494 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:18.104496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:18.104982 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:18.104984 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:18.105117 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:18.105749 (+ 632us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:36:18.105749 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:18.105780 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:36:18.105804 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:36:18.106022 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:36:18.106028 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:36:18.106031 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:36:18.106150 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 14:36:18.106258 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":120,"threads_started":1}
I20260811 14:36:18.107074 18455 heartbeater.cc:347] Connected to a master server at 127.2.103.126:37511
I20260811 14:36:18.107173 18455 heartbeater.cc:464] Registering TS with master...
I20260811 14:36:18.107386 18455 heartbeater.cc:511] Master 127.2.103.126:37511 requested a full tablet report, sending...
I20260811 14:36:18.107777 17927 ts_manager.cc:194] Registered new tserver with Master: 63a849306a734caf8af03b2a4f50fd61 (127.2.103.66:37257)
I20260811 14:36:18.108172 17927 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.103.66:40969
W20260811 14:36:18.192698 18458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:36:18.192983 18458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:36:18.193022 18458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:36:18.194213 18458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:36:18.194280 18458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:36:18.194320 18458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:36:18.195748 18458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:38713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:37511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:36:18.196051 18458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:36:18.196390 18458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:36:18.201959 18465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:18.202049 18468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:18.201977 18466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:18.202298 18458 server_base.cc:1034] running on GCE node
I20260811 14:36:18.202526 18458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:36:18.202809 18458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:36:18.203991 18458 hybrid_clock.cc:648] HybridClock initialized: now 1786458978203993 us; error 47 us; skew 500 ppm
I20260811 14:36:18.205387 18458 webserver.cc:460] Webserver started at http://127.2.103.67:34093/ using document root <none> and password file <none>
I20260811 14:36:18.205672 18458 fs_manager.cc:359] Metadata directory not provided
I20260811 14:36:18.205745 18458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:36:18.205909 18458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:36:18.207219 18458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "ef558a22f3e243f584c9fca4b41ff0d4"
format_stamp: "Formatted at 2026-08-11 14:36:18 on dist-test-slave-twwt"
server_key: "1011a15488f8318e7467ff5b0b886bdd"
server_key_iv: "1459f4d2f8c3520160139fe7f6e0d16e"
server_key_version: "encryptionkey@0"
I20260811 14:36:18.207605 18458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "ef558a22f3e243f584c9fca4b41ff0d4"
format_stamp: "Formatted at 2026-08-11 14:36:18 on dist-test-slave-twwt"
server_key: "1011a15488f8318e7467ff5b0b886bdd"
server_key_iv: "1459f4d2f8c3520160139fe7f6e0d16e"
server_key_version: "encryptionkey@0"
I20260811 14:36:18.208998 18458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:36:18.209777 18481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:18.209982 18458 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:36:18.210095 18458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "ef558a22f3e243f584c9fca4b41ff0d4"
format_stamp: "Formatted at 2026-08-11 14:36:18 on dist-test-slave-twwt"
server_key: "1011a15488f8318e7467ff5b0b886bdd"
server_key_iv: "1459f4d2f8c3520160139fe7f6e0d16e"
server_key_version: "encryptionkey@0"
I20260811 14:36:18.210173 18458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:36:18.225796 18458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:36:18.226150 18458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:36:18.226315 18458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:36:18.226570 18458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:36:18.226958 18458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:36:18.227016 18458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:18.227053 18458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:36:18.227100 18458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:18.233551 18458 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:40977
I20260811 14:36:18.233614 18668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:40977 every 8 connection(s)
I20260811 14:36:18.234038 18458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:36:18.239193 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:18.234858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:35169 (local address 127.2.103.126:37511)
0811 14:36:18.234943 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:18.234945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:18.235164 (+ 219us) server_negotiation.cc:407] Connection header received
0811 14:36:18.235647 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:18.235649 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:18.235659 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:18.235713 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:18.236374 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:18.236881 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:18.237845 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:18.238044 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:18.238096 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:18.238964 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:36:18.238965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:36:18.238966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:36:18.238970 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:36:18.238985 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:36:18.238989 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:36:18.239033 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:18.239108 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 14:36:18.239130 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 14:36:18.239401 18674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:18.234831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:37511 (local address 127.2.103.67:35169)
0811 14:36:18.235042 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:18.235077 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:18.235566 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:18.235767 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:18.235770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:18.235779 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:18.236304 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:18.236308 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:18.236986 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:18.236987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:18.237767 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:18.237768 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:18.238120 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:18.238121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:18.238229 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:18.238808 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:36:18.238808 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:18.238839 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:36:18.238879 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:36:18.239033 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:36:18.239036 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:36:18.239038 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:36:18.239085 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 14:36:18.239165 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":108,"threads_started":1}
I20260811 14:36:18.239971 18670 heartbeater.cc:347] Connected to a master server at 127.2.103.126:37511
I20260811 14:36:18.240087 18670 heartbeater.cc:464] Registering TS with master...
I20260811 14:36:18.240284 18670 heartbeater.cc:511] Master 127.2.103.126:37511 requested a full tablet report, sending...
I20260811 14:36:18.240669 17927 ts_manager.cc:194] Registered new tserver with Master: ef558a22f3e243f584c9fca4b41ff0d4 (127.2.103.67:40977)
I20260811 14:36:18.241070 17927 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.103.67:35169
I20260811 14:36:18.241133 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 18458
I20260811 14:36:18.241232 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:36:18.241466 2461 external_mini_cluster.cc:1428] Setting key 3a3b8b7ea2d21ba45e4dd57121a241f7
I20260811 14:36:18.242389 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 14:36:18.244235 17997 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:18.243706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35300 (local address 127.2.103.126:37511)
0811 14:36:18.243788 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:18.243790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:18.243911 (+ 121us) server_negotiation.cc:407] Connection header received
0811 14:36:18.243969 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:18.243970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:18.243981 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:18.244031 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:18.244158 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35300: BlockingRecv error: recv got EOF from 127.0.0.1:35300 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 14:36:18.244266 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 18026
I20260811 14:36:18.277279 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 18236
I20260811 14:36:18.291862 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 18458
I20260811 14:36:18.305910 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 17876
2026-08-11T14:36:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (749 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T14:36:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:36:18Z Disabled control of system clock
I20260811 14:36:18.348583 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:38559
--webserver_interface=127.2.103.126
--webserver_port=0
--builtin_ntp_servers=127.2.103.84:33541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:38559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 14:36:18.437615 18686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:36:18.437825 18686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:36:18.437853 18686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:36:18.438997 18686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:36:18.439043 18686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:36:18.439057 18686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:36:18.439069 18686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:36:18.439081 18686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:36:18.440407 18686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:33541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:38559
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:38559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:36:18.440622 18686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:36:18.440930 18686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:36:18.446013 18699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:18.446019 18693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:18.446108 18686 server_base.cc:1034] running on GCE node
W20260811 14:36:18.446048 18694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:18.446544 18686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:36:18.446930 18686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:36:18.448115 18686 hybrid_clock.cc:648] HybridClock initialized: now 1786458978448092 us; error 36 us; skew 500 ppm
I20260811 14:36:18.449644 18686 webserver.cc:460] Webserver started at http://127.2.103.126:41225/ using document root <none> and password file <none>
I20260811 14:36:18.449949 18686 fs_manager.cc:359] Metadata directory not provided
I20260811 14:36:18.450029 18686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:36:18.450198 18686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:36:18.451442 18686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "652b370f76a544bb8aea3868ee3ee34c"
format_stamp: "Formatted at 2026-08-11 14:36:18 on dist-test-slave-twwt"
server_key: "f4fd634c99b0ee7e0f1a0c986188b49c"
server_key_iv: "b11e44942f23f02ec6359ad9cf290dc5"
server_key_version: "encryptionkey@0"
I20260811 14:36:18.451830 18686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "652b370f76a544bb8aea3868ee3ee34c"
format_stamp: "Formatted at 2026-08-11 14:36:18 on dist-test-slave-twwt"
server_key: "f4fd634c99b0ee7e0f1a0c986188b49c"
server_key_iv: "b11e44942f23f02ec6359ad9cf290dc5"
server_key_version: "encryptionkey@0"
I20260811 14:36:18.453310 18686 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:36:18.454169 18709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:18.454370 18686 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:36:18.454497 18686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "652b370f76a544bb8aea3868ee3ee34c"
format_stamp: "Formatted at 2026-08-11 14:36:18 on dist-test-slave-twwt"
server_key: "f4fd634c99b0ee7e0f1a0c986188b49c"
server_key_iv: "b11e44942f23f02ec6359ad9cf290dc5"
server_key_version: "encryptionkey@0"
I20260811 14:36:18.454581 18686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:36:18.458210 18686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:36:18.458532 18686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:36:18.458710 18686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:36:18.462981 18686 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:38559
I20260811 14:36:18.463052 18792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:38559 every 8 connection(s)
I20260811 14:36:18.463440 18686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:36:18.464182 18793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:36:18.467248 18793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c: Bootstrap starting.
I20260811 14:36:18.467885 18793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c: Neither blocks nor log segments found. Creating new log.
I20260811 14:36:18.468209 18793 log.cc:824] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c: Log is configured to *not* fsync() on all Append() calls
I20260811 14:36:18.468997 18793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c: No bootstrap required, opened a new log
I20260811 14:36:18.470705 18793 raft_consensus.cc:348] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "652b370f76a544bb8aea3868ee3ee34c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38559 } }
I20260811 14:36:18.470854 18793 raft_consensus.cc:374] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:36:18.470878 18793 raft_consensus.cc:729] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652b370f76a544bb8aea3868ee3ee34c, State: Initialized, Role: FOLLOWER
I20260811 14:36:18.471057 18793 consensus_queue.cc:260] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "652b370f76a544bb8aea3868ee3ee34c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38559 } }
I20260811 14:36:18.471122 18793 raft_consensus.cc:388] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:36:18.471163 18793 raft_consensus.cc:482] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:36:18.471210 18793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:36:18.471781 18793 raft_consensus.cc:504] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "652b370f76a544bb8aea3868ee3ee34c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38559 } }
I20260811 14:36:18.471899 18793 leader_election.cc:302] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 652b370f76a544bb8aea3868ee3ee34c; no voters:
I20260811 14:36:18.472087 18793 leader_election.cc:288] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:36:18.472157 18796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:36:18.472343 18796 raft_consensus.cc:686] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c [term 1 LEADER]: Becoming Leader. State: Replica: 652b370f76a544bb8aea3868ee3ee34c, State: Running, Role: LEADER
I20260811 14:36:18.472381 18793 sys_catalog.cc:564] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:36:18.472446 18796 consensus_queue.cc:237] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "652b370f76a544bb8aea3868ee3ee34c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38559 } }
I20260811 14:36:18.472877 18800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 652b370f76a544bb8aea3868ee3ee34c. Latest consensus state: current_term: 1 leader_uuid: "652b370f76a544bb8aea3868ee3ee34c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "652b370f76a544bb8aea3868ee3ee34c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38559 } } }
I20260811 14:36:18.472998 18800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c [sys.catalog]: This master's current role is: LEADER
I20260811 14:36:18.473101 18797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "652b370f76a544bb8aea3868ee3ee34c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "652b370f76a544bb8aea3868ee3ee34c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38559 } } }
I20260811 14:36:18.473181 18797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c [sys.catalog]: This master's current role is: LEADER
I20260811 14:36:18.473248 18809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:36:18.473673 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 18686
I20260811 14:36:18.473783 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:36:18.473875 18809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:36:18.473984 2461 external_mini_cluster.cc:1428] Setting key ded74966b39ac454253026b24ba29eb6
I20260811 14:36:18.475724 18809 catalog_manager.cc:1347] Generated new cluster ID: 5527aed4ff114ee2ab71a301222b12c7
I20260811 14:36:18.475785 18809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:36:18.480361 18828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:18.474695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55820 (local address 127.2.103.126:38559)
0811 14:36:18.478788 (+ 4093us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:18.478796 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:18.478830 (+ 34us) server_negotiation.cc:407] Connection header received
0811 14:36:18.478940 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:18.478945 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:18.478979 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:18.479135 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:18.479832 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:36:18.479836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:36:18.479836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:36:18.479841 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:36:18.479889 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:36:18.479900 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:36:18.479973 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:18.480050 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 14:36:18.480206 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3974,"thread_start_us":69,"threads_started":1}
I20260811 14:36:18.503408 18809 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:36:18.503640 18809 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:36:18.510277 18809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 652b370f76a544bb8aea3868ee3ee34c: Generated new TSK 0
I20260811 14:36:18.510528 18809 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:36:18.516482 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38559
--builtin_ntp_servers=127.2.103.84:33541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 14:36:18.603097 18833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:36:18.603369 18833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:36:18.603411 18833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:36:18.604683 18833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:36:18.604753 18833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:36:18.604797 18833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:36:18.606257 18833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:33541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:36:18.606559 18833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:36:18.606920 18833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:36:18.612679 18843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:18.612761 18833 server_base.cc:1034] running on GCE node
W20260811 14:36:18.612812 18842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:18.612679 18845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:18.613102 18833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:36:18.613379 18833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:36:18.614538 18833 hybrid_clock.cc:648] HybridClock initialized: now 1786458978614513 us; error 37 us; skew 500 ppm
I20260811 14:36:18.615926 18833 webserver.cc:460] Webserver started at http://127.2.103.65:34513/ using document root <none> and password file <none>
I20260811 14:36:18.616209 18833 fs_manager.cc:359] Metadata directory not provided
I20260811 14:36:18.616267 18833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:36:18.616460 18833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:36:18.617614 18833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "6d29f64e5ed34646bfdd3ac71355487e"
format_stamp: "Formatted at 2026-08-11 14:36:18 on dist-test-slave-twwt"
server_key: "bc14aade84f2920eaa0240b89c0088b4"
server_key_iv: "77ce157da0b145d8efb61fd01b66dcfb"
server_key_version: "encryptionkey@0"
I20260811 14:36:18.617990 18833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "6d29f64e5ed34646bfdd3ac71355487e"
format_stamp: "Formatted at 2026-08-11 14:36:18 on dist-test-slave-twwt"
server_key: "bc14aade84f2920eaa0240b89c0088b4"
server_key_iv: "77ce157da0b145d8efb61fd01b66dcfb"
server_key_version: "encryptionkey@0"
I20260811 14:36:18.619498 18833 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:36:18.620278 18852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:18.620479 18833 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:36:18.620576 18833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "6d29f64e5ed34646bfdd3ac71355487e"
format_stamp: "Formatted at 2026-08-11 14:36:18 on dist-test-slave-twwt"
server_key: "bc14aade84f2920eaa0240b89c0088b4"
server_key_iv: "77ce157da0b145d8efb61fd01b66dcfb"
server_key_version: "encryptionkey@0"
I20260811 14:36:18.620656 18833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:36:18.624293 18833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:36:18.624565 18833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:36:18.624701 18833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:36:18.624922 18833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:36:18.625241 18833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:36:18.625293 18833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:18.625313 18833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:36:18.625360 18833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:18.631495 18833 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:44871
I20260811 14:36:18.631588 19012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:44871 every 8 connection(s)
I20260811 14:36:18.631915 18833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:36:18.632879 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 18833
I20260811 14:36:18.633002 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:36:18.633208 2461 external_mini_cluster.cc:1428] Setting key 963e80f4aed8b82480286a92b62aa29e
I20260811 14:36:18.634193 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38559
--builtin_ntp_servers=127.2.103.84:33541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 14:36:18.635421 18828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:18.632664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:60077 (local address 127.2.103.126:38559)
0811 14:36:18.632724 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:18.632727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:18.633029 (+ 302us) server_negotiation.cc:407] Connection header received
0811 14:36:18.633708 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:18.633709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:18.633716 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:18.633781 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:18.635176 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:36:18.635178 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:36:18.635179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:36:18.635183 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:36:18.635200 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:36:18.635205 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:36:18.635259 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:18.635335 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 14:36:18.635350 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:36:18.635667 19018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:18.632633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:38559 (local address 127.2.103.65:60077)
0811 14:36:18.632898 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:18.632932 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:18.633601 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:18.633851 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:18.633855 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:18.633866 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:18.634991 (+ 1125us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:36:18.634991 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:18.635027 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:36:18.635058 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:36:18.635255 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:36:18.635261 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:36:18.635273 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 14:36:18.635349 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 14:36:18.635411 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":110,"threads_started":1}
I20260811 14:36:18.636124 19014 heartbeater.cc:347] Connected to a master server at 127.2.103.126:38559
I20260811 14:36:18.636233 19014 heartbeater.cc:464] Registering TS with master...
I20260811 14:36:18.636395 19014 heartbeater.cc:511] Master 127.2.103.126:38559 requested a full tablet report, sending...
I20260811 14:36:18.636905 18738 ts_manager.cc:194] Registered new tserver with Master: 6d29f64e5ed34646bfdd3ac71355487e (127.2.103.65:44871)
W20260811 14:36:18.720192 19019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:36:18.720412 19019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:36:18.720435 19019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:36:18.721659 19019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:36:18.721705 19019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:36:18.721731 19019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:36:18.723201 19019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:33541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:36:18.723477 19019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:36:18.723798 19019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:36:18.728935 19028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:18.728972 19030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:18.728963 19033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:18.729100 19019 server_base.cc:1034] running on GCE node
I20260811 14:36:18.729382 19019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:36:18.729665 19019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:36:18.730834 19019 hybrid_clock.cc:648] HybridClock initialized: now 1786458978730832 us; error 72 us; skew 500 ppm
I20260811 14:36:18.732271 19019 webserver.cc:460] Webserver started at http://127.2.103.66:44957/ using document root <none> and password file <none>
I20260811 14:36:18.732522 19019 fs_manager.cc:359] Metadata directory not provided
I20260811 14:36:18.732595 19019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:36:18.732774 19019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:36:18.733886 19019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "df0095d6dfd546d3ad5bb9f5365ccf1b"
format_stamp: "Formatted at 2026-08-11 14:36:18 on dist-test-slave-twwt"
server_key: "68bb2579d0d33484dea9f11793c932e4"
server_key_iv: "15fbf5498d86581ce3bad6be8b18283e"
server_key_version: "encryptionkey@0"
I20260811 14:36:18.734269 19019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "df0095d6dfd546d3ad5bb9f5365ccf1b"
format_stamp: "Formatted at 2026-08-11 14:36:18 on dist-test-slave-twwt"
server_key: "68bb2579d0d33484dea9f11793c932e4"
server_key_iv: "15fbf5498d86581ce3bad6be8b18283e"
server_key_version: "encryptionkey@0"
I20260811 14:36:18.735723 19019 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:36:18.736506 19049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:18.736682 19019 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 14:36:18.736785 19019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "df0095d6dfd546d3ad5bb9f5365ccf1b"
format_stamp: "Formatted at 2026-08-11 14:36:18 on dist-test-slave-twwt"
server_key: "68bb2579d0d33484dea9f11793c932e4"
server_key_iv: "15fbf5498d86581ce3bad6be8b18283e"
server_key_version: "encryptionkey@0"
I20260811 14:36:18.736873 19019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:36:18.740507 19019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:36:18.740767 19019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:36:18.740916 19019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:36:18.741137 19019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:36:18.741459 19019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:36:18.741514 19019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:18.741554 19019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:36:18.741580 19019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:18.747516 19019 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:35783
I20260811 14:36:18.747562 19223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:35783 every 8 connection(s)
I20260811 14:36:18.747902 19019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:36:18.750417 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 19019
I20260811 14:36:18.750551 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:36:18.750798 2461 external_mini_cluster.cc:1428] Setting key 42910f53faf91eaef483db3db9e318ce
I20260811 14:36:18.751339 18828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:18.748666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:48661 (local address 127.2.103.126:38559)
0811 14:36:18.748730 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:18.748732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:18.748965 (+ 233us) server_negotiation.cc:407] Connection header received
0811 14:36:18.749421 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:18.749422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:18.749428 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:18.749487 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:18.751029 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:36:18.751031 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:36:18.751032 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:36:18.751040 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:36:18.751064 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:36:18.751070 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:36:18.751145 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:18.751252 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 14:36:18.751267 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 14:36:18.751534 19228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:18.748648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:38559 (local address 127.2.103.66:48661)
0811 14:36:18.748831 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:18.748887 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:18.749334 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:18.749550 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:18.749553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:18.749561 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:18.750790 (+ 1229us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:36:18.750790 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:18.750831 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:36:18.750881 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:36:18.751152 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:36:18.751159 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:36:18.751162 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:36:18.751244 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 14:36:18.751305 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260811 14:36:18.751986 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38559
--builtin_ntp_servers=127.2.103.84:33541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 14:36:18.751973 19225 heartbeater.cc:347] Connected to a master server at 127.2.103.126:38559
I20260811 14:36:18.752094 19225 heartbeater.cc:464] Registering TS with master...
I20260811 14:36:18.752246 19225 heartbeater.cc:511] Master 127.2.103.126:38559 requested a full tablet report, sending...
I20260811 14:36:18.752671 18738 ts_manager.cc:194] Registered new tserver with Master: df0095d6dfd546d3ad5bb9f5365ccf1b (127.2.103.66:35783)
W20260811 14:36:18.837975 19231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:36:18.838181 19231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:36:18.838203 19231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:36:18.839461 19231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:36:18.839504 19231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:36:18.839555 19231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:36:18.841040 19231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:33541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:36:18.841297 19231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:36:18.841611 19231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:36:18.846868 19248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:18.846868 19242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:18.846913 19246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:18.847041 19231 server_base.cc:1034] running on GCE node
I20260811 14:36:18.847340 19231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:36:18.847666 19231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:36:18.848836 19231 hybrid_clock.cc:648] HybridClock initialized: now 1786458978848808 us; error 39 us; skew 500 ppm
I20260811 14:36:18.850098 19231 webserver.cc:460] Webserver started at http://127.2.103.67:41419/ using document root <none> and password file <none>
I20260811 14:36:18.850383 19231 fs_manager.cc:359] Metadata directory not provided
I20260811 14:36:18.850471 19231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:36:18.850678 19231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:36:18.851897 19231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "1a865e8482bf4588b1d5218ba9407904"
format_stamp: "Formatted at 2026-08-11 14:36:18 on dist-test-slave-twwt"
server_key: "f9adc172ea13f36d1fde7e25a2e60c95"
server_key_iv: "a05ce2601be84b805e9b1002691c5ecc"
server_key_version: "encryptionkey@0"
I20260811 14:36:18.852293 19231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "1a865e8482bf4588b1d5218ba9407904"
format_stamp: "Formatted at 2026-08-11 14:36:18 on dist-test-slave-twwt"
server_key: "f9adc172ea13f36d1fde7e25a2e60c95"
server_key_iv: "a05ce2601be84b805e9b1002691c5ecc"
server_key_version: "encryptionkey@0"
I20260811 14:36:18.853744 19231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:36:18.854494 19255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:18.854697 19231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:36:18.854797 19231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "1a865e8482bf4588b1d5218ba9407904"
format_stamp: "Formatted at 2026-08-11 14:36:18 on dist-test-slave-twwt"
server_key: "f9adc172ea13f36d1fde7e25a2e60c95"
server_key_iv: "a05ce2601be84b805e9b1002691c5ecc"
server_key_version: "encryptionkey@0"
I20260811 14:36:18.854887 19231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:36:18.858466 19231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:36:18.858781 19231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:36:18.858932 19231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:36:18.859159 19231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:36:18.859488 19231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:36:18.859543 19231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:18.859582 19231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:36:18.859611 19231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:18.865499 19231 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:39547
I20260811 14:36:18.865541 19423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:39547 every 8 connection(s)
I20260811 14:36:18.865921 19231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:36:18.868016 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 19231
I20260811 14:36:18.868101 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:36:18.868304 2461 external_mini_cluster.cc:1428] Setting key d387eb58c039d94735f4540f88cc26bf
I20260811 14:36:18.869486 18828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:18.866725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:38375 (local address 127.2.103.126:38559)
0811 14:36:18.866802 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:18.866805 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:18.867130 (+ 325us) server_negotiation.cc:407] Connection header received
0811 14:36:18.867635 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:18.867636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:18.867645 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:18.867700 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:18.869197 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:36:18.869199 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:36:18.869199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:36:18.869203 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:36:18.869221 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:36:18.869226 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:36:18.869285 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:18.869394 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 14:36:18.869414 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 14:36:18.869725 19430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:18.866736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:38559 (local address 127.2.103.67:38375)
0811 14:36:18.866995 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:18.867031 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:18.867544 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:18.867757 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:18.867760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:18.867769 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:18.869012 (+ 1243us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:36:18.869013 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:18.869054 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:36:18.869100 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:36:18.869299 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:36:18.869308 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:36:18.869316 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 14:36:18.869411 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 14:36:18.869478 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":99,"threads_started":1}
I20260811 14:36:18.870134 19424 heartbeater.cc:347] Connected to a master server at 127.2.103.126:38559
I20260811 14:36:18.870249 19424 heartbeater.cc:464] Registering TS with master...
I20260811 14:36:18.870409 19424 heartbeater.cc:511] Master 127.2.103.126:38559 requested a full tablet report, sending...
I20260811 14:36:18.870878 18738 ts_manager.cc:194] Registered new tserver with Master: 1a865e8482bf4588b1d5218ba9407904 (127.2.103.67:39547)
I20260811 14:36:18.880028 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 14:36:18.882076 18828 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:36:18.881518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55836 (local address 127.2.103.126:38559)
0811 14:36:18.881576 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:18.881578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:18.881682 (+ 104us) server_negotiation.cc:407] Connection header received
0811 14:36:18.881728 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:18.881729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:18.881734 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:18.881785 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:18.881998 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55836: BlockingRecv error: recv got EOF from 127.0.0.1:55836 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 14:36:18.882176 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 18833
I20260811 14:36:18.905730 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 19019
I20260811 14:36:18.923360 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 19231
I20260811 14:36:18.941679 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 18686
2026-08-11T14:36:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (631 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:36:19 dist-test-slave-twwt krb5kdc[19440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:36:19 dist-test-slave-twwt krb5kdc[19440](info): set up 2 sockets
Aug 11 14:36:19 dist-test-slave-twwt krb5kdc[19440](info): commencing operation
krb5kdc: starting...
W20260811 14:36:20.228871 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:36:20 dist-test-slave-twwt krb5kdc[19440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:36:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:36:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:36:20.366566 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:32901
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:45829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:32901
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:36:20.455838 19457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:36:20.456069 19457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:36:20.456095 19457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:36:20.457324 19457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:36:20.457370 19457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:36:20.457386 19457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:36:20.457397 19457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:36:20.457410 19457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:36:20.458998 19457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:45829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:32901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:32901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:36:20.459280 19457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:36:20.459522 19457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:36:20.465142 19466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:20.465094 19465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:20.465526 19457 server_base.cc:1034] running on GCE node
W20260811 14:36:20.465374 19469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:20.465903 19457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:36:20.466274 19457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:36:20.467455 19457 hybrid_clock.cc:648] HybridClock initialized: now 1786458980467452 us; error 44 us; skew 500 ppm
Aug 11 14:36:20 dist-test-slave-twwt krb5kdc[19440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458980, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:36:20.470268 19457 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:36:20.470796 19457 webserver.cc:460] Webserver started at http://127.2.103.126:36223/ using document root <none> and password file <none>
I20260811 14:36:20.471045 19457 fs_manager.cc:359] Metadata directory not provided
I20260811 14:36:20.471114 19457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:36:20.471256 19457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:36:20.472427 19457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "953eff2253334f64a8446730488b0029"
format_stamp: "Formatted at 2026-08-11 14:36:20 on dist-test-slave-twwt"
server_key: "7921dc3008716cb29a50d72d076d8feb"
server_key_iv: "751fa14ef9cfb22ec0840da1655c9453"
server_key_version: "encryptionkey@0"
I20260811 14:36:20.472856 19457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "953eff2253334f64a8446730488b0029"
format_stamp: "Formatted at 2026-08-11 14:36:20 on dist-test-slave-twwt"
server_key: "7921dc3008716cb29a50d72d076d8feb"
server_key_iv: "751fa14ef9cfb22ec0840da1655c9453"
server_key_version: "encryptionkey@0"
I20260811 14:36:20.474254 19457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:36:20.475230 19483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:20.475467 19457 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:36:20.475593 19457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "953eff2253334f64a8446730488b0029"
format_stamp: "Formatted at 2026-08-11 14:36:20 on dist-test-slave-twwt"
server_key: "7921dc3008716cb29a50d72d076d8feb"
server_key_iv: "751fa14ef9cfb22ec0840da1655c9453"
server_key_version: "encryptionkey@0"
I20260811 14:36:20.475679 19457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:36:20.521796 19457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:36:20.524379 19457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:36:20.524601 19457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:36:20.529546 19457 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:32901
I20260811 14:36:20.529587 19561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:32901 every 8 connection(s)
I20260811 14:36:20.530031 19457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:36:20.531309 19565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:36:20.533670 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 19457
I20260811 14:36:20.533797 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:36:20.534008 2461 external_mini_cluster.cc:1428] Setting key 530bf61a225b4698b07afd072d47a5c1
I20260811 14:36:20.534305 19565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029: Bootstrap starting.
I20260811 14:36:20.535120 19565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029: Neither blocks nor log segments found. Creating new log.
I20260811 14:36:20.535450 19565 log.cc:824] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029: Log is configured to *not* fsync() on all Append() calls
I20260811 14:36:20.536336 19565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029: No bootstrap required, opened a new log
I20260811 14:36:20.538213 19565 raft_consensus.cc:348] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "953eff2253334f64a8446730488b0029" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 32901 } }
I20260811 14:36:20.538386 19565 raft_consensus.cc:374] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:36:20.538437 19565 raft_consensus.cc:729] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953eff2253334f64a8446730488b0029, State: Initialized, Role: FOLLOWER
I20260811 14:36:20.538578 19565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "953eff2253334f64a8446730488b0029" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 32901 } }
I20260811 14:36:20.538733 19565 raft_consensus.cc:388] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:36:20.538789 19565 raft_consensus.cc:482] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:36:20.538851 19565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:36:20.539620 19565 raft_consensus.cc:504] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "953eff2253334f64a8446730488b0029" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 32901 } }
I20260811 14:36:20.539741 19565 leader_election.cc:302] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 953eff2253334f64a8446730488b0029; no voters:
I20260811 14:36:20.539944 19565 leader_election.cc:288] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:36:20.540030 19578 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:36:20.540218 19578 raft_consensus.cc:686] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029 [term 1 LEADER]: Becoming Leader. State: Replica: 953eff2253334f64a8446730488b0029, State: Running, Role: LEADER
I20260811 14:36:20.540264 19565 sys_catalog.cc:564] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:36:20.540342 19578 consensus_queue.cc:237] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "953eff2253334f64a8446730488b0029" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 32901 } }
I20260811 14:36:20.540714 19578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 953eff2253334f64a8446730488b0029. Latest consensus state: current_term: 1 leader_uuid: "953eff2253334f64a8446730488b0029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "953eff2253334f64a8446730488b0029" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 32901 } } }
I20260811 14:36:20.540728 19579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "953eff2253334f64a8446730488b0029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "953eff2253334f64a8446730488b0029" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 32901 } } }
I20260811 14:36:20.540808 19578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029 [sys.catalog]: This master's current role is: LEADER
I20260811 14:36:20.540838 19579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029 [sys.catalog]: This master's current role is: LEADER
Aug 11 14:36:20 dist-test-slave-twwt krb5kdc[19440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
W20260811 14:36:20.542913 19601 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:36:20.542980 19601 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:36:20.543092 19604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:36:20.543272 19604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:36:20.546422 19574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:20.534656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52278 (local address 127.2.103.126:32901)
0811 14:36:20.534879 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:20.534889 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:20.534928 (+ 39us) server_negotiation.cc:407] Connection header received
0811 14:36:20.535478 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:20.535487 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:20.535579 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:20.535752 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:20.536778 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:20.537816 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:20.538850 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:20.539499 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:20.539630 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:20.542758 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:36:20.542763 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:36:20.542763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:36:20.542777 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:36:20.544641 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:36:20.545086 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:36:20.545089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:36:20.545089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:36:20.545178 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:36:20.545452 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:36:20.545454 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:36:20.545454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:36:20.545720 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:36:20.545838 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:20.546000 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 14:36:20.546117 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":103,"threads_started":1}
I20260811 14:36:20.548092 19604 catalog_manager.cc:1347] Generated new cluster ID: b9de8742bc1a449585f438a48491393d
I20260811 14:36:20.548158 19604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:36:20.557374 19604 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:36:20.558231 19604 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:36:20.565699 19604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029: Generated new TSK 0
I20260811 14:36:20.565961 19604 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
I20260811 14:36:20.598636 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:32901
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:45829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:36:20.685894 19614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:36:20.686142 19614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:36:20.686187 19614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:36:20.687466 19614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:36:20.687538 19614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:36:20.687587 19614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:36:20.689162 19614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:45829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:32901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:36:20.689446 19614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:36:20.689713 19614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:36:20.696244 19628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:20.696112 19624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:20.696116 19625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:20.696599 19614 server_base.cc:1034] running on GCE node
I20260811 14:36:20.696902 19614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:36:20.697180 19614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:36:20.698383 19614 hybrid_clock.cc:648] HybridClock initialized: now 1786458980698356 us; error 44 us; skew 500 ppm
Aug 11 14:36:20 dist-test-slave-twwt krb5kdc[19440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458980, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:36:20.701401 19614 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:36:20.701985 19614 webserver.cc:460] Webserver started at http://127.2.103.65:36583/ using document root <none> and password file <none>
I20260811 14:36:20.702270 19614 fs_manager.cc:359] Metadata directory not provided
I20260811 14:36:20.702346 19614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:36:20.702509 19614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:36:20.703694 19614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "fd476598415740c0b44492d4be66575c"
format_stamp: "Formatted at 2026-08-11 14:36:20 on dist-test-slave-twwt"
server_key: "8348d8f37b8d4ca155985858e87dd8ef"
server_key_iv: "667a3e924ff43e3ede4e40919c197564"
server_key_version: "encryptionkey@0"
I20260811 14:36:20.704113 19614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "fd476598415740c0b44492d4be66575c"
format_stamp: "Formatted at 2026-08-11 14:36:20 on dist-test-slave-twwt"
server_key: "8348d8f37b8d4ca155985858e87dd8ef"
server_key_iv: "667a3e924ff43e3ede4e40919c197564"
server_key_version: "encryptionkey@0"
I20260811 14:36:20.705681 19614 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:36:20.706691 19639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:20.706971 19614 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:36:20.707090 19614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "fd476598415740c0b44492d4be66575c"
format_stamp: "Formatted at 2026-08-11 14:36:20 on dist-test-slave-twwt"
server_key: "8348d8f37b8d4ca155985858e87dd8ef"
server_key_iv: "667a3e924ff43e3ede4e40919c197564"
server_key_version: "encryptionkey@0"
I20260811 14:36:20.707176 19614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:36:20.717092 19614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:36:20.719623 19614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:36:20.719856 19614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:36:20.720166 19614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:36:20.720531 19614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:36:20.720589 19614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:20.720626 19614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:36:20.720674 19614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:20.727164 19614 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:39187
I20260811 14:36:20.727210 19823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:39187 every 8 connection(s)
I20260811 14:36:20.727566 19614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:36:20 dist-test-slave-twwt krb5kdc[19440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458980, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:36:20.735484 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 19614
I20260811 14:36:20.735584 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:36:20.735826 2461 external_mini_cluster.cc:1428] Setting key a962f2d951a7668b7fb27272c257f2c5
I20260811 14:36:20.738364 19574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:20.728380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:46851 (local address 127.2.103.126:32901)
0811 14:36:20.728432 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:20.728434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:20.728675 (+ 241us) server_negotiation.cc:407] Connection header received
0811 14:36:20.729186 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:20.729187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:20.729198 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:20.729266 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:20.730376 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:20.730961 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:20.731991 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:20.732215 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:20.732275 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:20.734845 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:36:20.734846 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:36:20.734847 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:36:20.734854 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:36:20.736758 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:36:20.737181 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:36:20.737182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:36:20.737183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:36:20.737260 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:36:20.737489 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:36:20.737490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:36:20.737490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:36:20.737665 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:36:20.737755 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:20.738089 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 14:36:20.738205 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 14:36:20.738384 19828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:20.728341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:32901 (local address 127.2.103.65:46851)
0811 14:36:20.728538 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:20.728583 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:20.729095 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:20.729346 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:20.729350 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:20.729980 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:20.730212 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:20.730226 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:20.731061 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:20.731062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:20.731902 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:20.731905 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:20.732302 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:20.732303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:20.732406 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:20.733014 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:20.733024 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:20.734733 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:36:20.736847 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:36:20.736854 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:36:20.736856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:36:20.737099 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:36:20.737329 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:36:20.737329 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:36:20.737329 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:36:20.737423 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:36:20.737767 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:36:20.737775 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:36:20.737931 (+ 156us) client_negotiation.cc:769] Sending connection context
0811 14:36:20.738038 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 14:36:20.738188 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":113,"threads_started":1}
I20260811 14:36:20.739068 19824 heartbeater.cc:347] Connected to a master server at 127.2.103.126:32901
I20260811 14:36:20.739183 19824 heartbeater.cc:464] Registering TS with master...
I20260811 14:36:20.739428 19824 heartbeater.cc:511] Master 127.2.103.126:32901 requested a full tablet report, sending...
I20260811 14:36:20.740037 19507 ts_manager.cc:194] Registered new tserver with Master: fd476598415740c0b44492d4be66575c (127.2.103.65:39187)
I20260811 14:36:20.741078 19507 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:46851
WARNING: no policy specified for kudu/127.2.103.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
I20260811 14:36:20.766592 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:32901
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:45829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:36:20.855484 19831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:36:20.855754 19831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:36:20.855796 19831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:36:20.856998 19831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:36:20.857072 19831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:36:20.857116 19831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:36:20.858686 19831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:45829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:32901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:36:20.858983 19831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:36:20.859238 19831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:36:20.865126 19839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:20.865175 19840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:20.865240 19843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:20.865609 19831 server_base.cc:1034] running on GCE node
I20260811 14:36:20.865850 19831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:36:20.866088 19831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:36:20.867266 19831 hybrid_clock.cc:648] HybridClock initialized: now 1786458980867260 us; error 39 us; skew 500 ppm
Aug 11 14:36:20 dist-test-slave-twwt krb5kdc[19440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458980, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:36:20.870298 19831 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:36:20.870910 19831 webserver.cc:460] Webserver started at http://127.2.103.66:46121/ using document root <none> and password file <none>
I20260811 14:36:20.871181 19831 fs_manager.cc:359] Metadata directory not provided
I20260811 14:36:20.871260 19831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:36:20.871461 19831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:36:20.872681 19831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "6bbcd69d55374b72a02b0da00c2ec365"
format_stamp: "Formatted at 2026-08-11 14:36:20 on dist-test-slave-twwt"
server_key: "6b00276912e4ad94762a24108ec96d30"
server_key_iv: "82d4c60f33f7079232e31d4ccf36bf72"
server_key_version: "encryptionkey@0"
I20260811 14:36:20.873131 19831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "6bbcd69d55374b72a02b0da00c2ec365"
format_stamp: "Formatted at 2026-08-11 14:36:20 on dist-test-slave-twwt"
server_key: "6b00276912e4ad94762a24108ec96d30"
server_key_iv: "82d4c60f33f7079232e31d4ccf36bf72"
server_key_version: "encryptionkey@0"
I20260811 14:36:20.874724 19831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260811 14:36:20.875586 19857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:20.875787 19831 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:36:20.875914 19831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "6bbcd69d55374b72a02b0da00c2ec365"
format_stamp: "Formatted at 2026-08-11 14:36:20 on dist-test-slave-twwt"
server_key: "6b00276912e4ad94762a24108ec96d30"
server_key_iv: "82d4c60f33f7079232e31d4ccf36bf72"
server_key_version: "encryptionkey@0"
I20260811 14:36:20.876000 19831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:36:20.884516 19831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:36:20.887190 19831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:36:20.887392 19831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:36:20.887665 19831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:36:20.888015 19831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:36:20.888075 19831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:20.888123 19831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:36:20.888155 19831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:20.895015 19831 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:32821
I20260811 14:36:20.895056 20035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:32821 every 8 connection(s)
I20260811 14:36:20.895452 19831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
Aug 11 14:36:20 dist-test-slave-twwt krb5kdc[19440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458980, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:36:20.903265 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 19831
I20260811 14:36:20.903381 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:36:20.903645 2461 external_mini_cluster.cc:1428] Setting key 412a0d4338ce87be5c000e3aa4e3471a
I20260811 14:36:20.906163 19574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:20.896324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:43419 (local address 127.2.103.126:32901)
0811 14:36:20.896415 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:20.896418 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:20.896629 (+ 211us) server_negotiation.cc:407] Connection header received
0811 14:36:20.897219 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:20.897220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:20.897232 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:20.897296 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:20.898393 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:20.899034 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:20.900299 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:20.900504 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:20.900567 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:20.902998 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:36:20.903000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:36:20.903000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:36:20.903007 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:36:20.904581 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:36:20.905116 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:36:20.905117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:36:20.905117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:36:20.905168 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:36:20.905487 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:36:20.905488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:36:20.905488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:36:20.905596 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:36:20.905650 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:20.906028 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 14:36:20.906098 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 14:36:20.906554 20041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:20.896261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:32901 (local address 127.2.103.66:43419)
0811 14:36:20.896465 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:20.896508 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:20.897083 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:20.897369 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:20.897372 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:20.898060 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:20.898286 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:20.898291 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:20.899125 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:20.899126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:20.900203 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:20.900207 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:20.900590 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:20.900591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:20.900700 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:20.901326 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:20.901336 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:20.902909 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:36:20.904663 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:36:20.904671 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:36:20.904673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:36:20.905032 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:36:20.905238 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:36:20.905239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:36:20.905239 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:36:20.905415 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:36:20.905667 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:36:20.905674 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:36:20.905880 (+ 206us) client_negotiation.cc:769] Sending connection context
0811 14:36:20.906011 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 14:36:20.906214 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":113,"threads_started":1}
I20260811 14:36:20.907094 20037 heartbeater.cc:347] Connected to a master server at 127.2.103.126:32901
I20260811 14:36:20.907219 20037 heartbeater.cc:464] Registering TS with master...
I20260811 14:36:20.907446 20037 heartbeater.cc:511] Master 127.2.103.126:32901 requested a full tablet report, sending...
I20260811 14:36:20.907862 19507 ts_manager.cc:194] Registered new tserver with Master: 6bbcd69d55374b72a02b0da00c2ec365 (127.2.103.66:32821)
I20260811 14:36:20.908280 19507 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:43419
WARNING: no policy specified for kudu/127.2.103.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
I20260811 14:36:20.934551 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:32901
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:45829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:36:21.021531 20054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:36:21.021798 20054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:36:21.021837 20054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:36:21.023139 20054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:36:21.023211 20054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:36:21.023254 20054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:36:21.024808 20054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:45829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:32901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:36:21.025117 20054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:36:21.025373 20054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:36:21.031180 20066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:21.031319 20054 server_base.cc:1034] running on GCE node
W20260811 14:36:21.031190 20063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:36:21.031250 20062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:36:21.031662 20054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:36:21.031926 20054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:36:21.033133 20054 hybrid_clock.cc:648] HybridClock initialized: now 1786458981033125 us; error 38 us; skew 500 ppm
Aug 11 14:36:21 dist-test-slave-twwt krb5kdc[19440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458981, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:36:21.036029 20054 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:36:21.036564 20054 webserver.cc:460] Webserver started at http://127.2.103.67:34707/ using document root <none> and password file <none>
I20260811 14:36:21.036815 20054 fs_manager.cc:359] Metadata directory not provided
I20260811 14:36:21.036887 20054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:36:21.037065 20054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:36:21.038203 20054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "e3f9596e08ca467daf1f763e6fb669a4"
format_stamp: "Formatted at 2026-08-11 14:36:21 on dist-test-slave-twwt"
server_key: "5c7dcbf28676e803c2d97b4d3b389b2d"
server_key_iv: "f372f55c208987bc6bf884afb131127c"
server_key_version: "encryptionkey@0"
I20260811 14:36:21.038592 20054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "e3f9596e08ca467daf1f763e6fb669a4"
format_stamp: "Formatted at 2026-08-11 14:36:21 on dist-test-slave-twwt"
server_key: "5c7dcbf28676e803c2d97b4d3b389b2d"
server_key_iv: "f372f55c208987bc6bf884afb131127c"
server_key_version: "encryptionkey@0"
I20260811 14:36:21.040091 20054 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:36:21.041059 20074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:21.041235 20054 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:36:21.041349 20054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "e3f9596e08ca467daf1f763e6fb669a4"
format_stamp: "Formatted at 2026-08-11 14:36:21 on dist-test-slave-twwt"
server_key: "5c7dcbf28676e803c2d97b4d3b389b2d"
server_key_iv: "f372f55c208987bc6bf884afb131127c"
server_key_version: "encryptionkey@0"
I20260811 14:36:21.041440 20054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:36:21.064569 20054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:36:21.066974 20054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:36:21.067167 20054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:36:21.067451 20054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:36:21.067821 20054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:36:21.067883 20054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:21.067951 20054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:36:21.067997 20054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:36:21.074801 20054 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:39979
I20260811 14:36:21.074860 20242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:39979 every 8 connection(s)
I20260811 14:36:21.075284 20054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:36:21.081367 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 20054
I20260811 14:36:21.081452 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:36:21.081657 2461 external_mini_cluster.cc:1428] Setting key 7657e1d8ac5cc229e8f351671112b107
Aug 11 14:36:21 dist-test-slave-twwt krb5kdc[19440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458981, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:36:21.085438 19574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:21.076203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:34169 (local address 127.2.103.126:32901)
0811 14:36:21.076268 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:21.076270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:21.076507 (+ 237us) server_negotiation.cc:407] Connection header received
0811 14:36:21.077104 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:21.077106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:21.077119 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:21.077176 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:21.078215 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.078799 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.079967 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.080183 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.080263 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:21.082711 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:36:21.082713 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:36:21.082713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:36:21.082720 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:36:21.084001 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:36:21.084424 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:36:21.084425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:36:21.084426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:36:21.084491 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:36:21.084756 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:36:21.084757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:36:21.084757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:36:21.084877 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:36:21.084957 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:21.085239 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 14:36:21.085347 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 14:36:21.085608 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:21.076173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:32901 (local address 127.2.103.67:34169)
0811 14:36:21.076372 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:21.076410 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:21.076986 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:21.077267 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:21.077270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:21.077906 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:36:21.078105 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:21.078110 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.078885 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.078886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:21.079825 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:21.079828 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.080271 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.080272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:21.080404 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:21.080984 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:36:21.080993 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:36:21.082585 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:36:21.084106 (+ 1521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:36:21.084113 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:36:21.084115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:36:21.084345 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:36:21.084564 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:36:21.084565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:36:21.084566 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:36:21.084679 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:36:21.084947 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:36:21.084954 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:36:21.085097 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 14:36:21.085203 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 14:36:21.085367 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":103,"threads_started":1}
I20260811 14:36:21.086048 20243 heartbeater.cc:347] Connected to a master server at 127.2.103.126:32901
I20260811 14:36:21.086133 20243 heartbeater.cc:464] Registering TS with master...
I20260811 14:36:21.086341 20243 heartbeater.cc:511] Master 127.2.103.126:32901 requested a full tablet report, sending...
I20260811 14:36:21.086781 19507 ts_manager.cc:194] Registered new tserver with Master: e3f9596e08ca467daf1f763e6fb669a4 (127.2.103.67:39979)
I20260811 14:36:21.087162 19507 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:34169
I20260811 14:36:21.093627 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:36:21.100975 19574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:21.095052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52280 (local address 127.2.103.126:32901)
0811 14:36:21.095119 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:21.095121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:21.095204 (+ 83us) server_negotiation.cc:407] Connection header received
0811 14:36:21.095245 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:21.095246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:21.095255 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:21.095305 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:36:21.095818 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.096299 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.096975 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.097153 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.097201 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:21.098448 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:36:21.098449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:36:21.098449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:36:21.098455 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:36:21.099795 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:36:21.100179 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:36:21.100181 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:36:21.100181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:36:21.100260 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:36:21.100481 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:36:21.100482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:36:21.100482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:36:21.100629 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:36:21.100709 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:21.100837 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 14:36:21.100907 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 14:36:21.102836 19507 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:36:21.104393 19507 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:36:21.111236 20261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:21.106892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:32821 (local address 127.0.0.1:43546)
0811 14:36:21.107218 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:21.107265 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:21.107379 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:21.107708 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:21.107709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:21.107711 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:36:21.107841 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:21.107844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.108787 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.108788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:21.109998 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:21.110001 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.110916 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.110917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:21.110997 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:21.111012 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 14:36:21.111087 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 14:36:21.111136 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":5,"thread_start_us":69,"threads_started":1}
I20260811 14:36:21.111254 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:21.106991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43546 (local address 127.2.103.66:32821)
0811 14:36:21.107152 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:21.107158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:21.107343 (+ 185us) server_negotiation.cc:407] Connection header received
0811 14:36:21.107504 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:21.107506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:21.107523 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:21.107583 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:36:21.107929 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.108675 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.110084 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.110826 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.110906 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:21.110967 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:21.111125 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 14:36:21.111170 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
I20260811 14:36:21.112016 20257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:21.106923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:39979 (local address 127.0.0.1:38456)
0811 14:36:21.107249 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:21.107266 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:21.107355 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:21.107649 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:21.107652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:21.107712 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:36:21.107814 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:21.107817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.109252 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.109253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:21.110330 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:21.110332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.111722 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.111724 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:21.111792 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:21.111795 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:36:21.111887 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 14:36:21.111940 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":4,"thread_start_us":92,"threads_started":1}
I20260811 14:36:21.112248 19940 tablet_service.cc:1467] Processing CreateTablet for tablet db473d335c0445829d9f27a985013587 (DEFAULT_TABLE table=test-table [id=fcdce2a4dacf41d7ade690efd6ba88c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:36:21.112406 20263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:21.107020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38456 (local address 127.2.103.67:39979)
0811 14:36:21.107203 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:21.107207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:21.107330 (+ 123us) server_negotiation.cc:407] Connection header received
0811 14:36:21.107464 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:21.107467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:21.107483 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:21.107535 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:36:21.107968 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.109137 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.110429 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.111260 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.111618 (+ 358us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:21.111686 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:21.112292 (+ 606us) server_negotiation.cc:299] Negotiation successful
0811 14:36:21.112336 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":85,"threads_started":1}
I20260811 14:36:21.112404 20265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:21.107382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58830 (local address 127.2.103.65:39187)
0811 14:36:21.107661 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:21.107666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:21.107676 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:36:21.107714 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:21.107716 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:21.107731 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:21.107806 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:36:21.108010 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.109136 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.111228 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.111949 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.112018 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:21.112070 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:21.112276 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 14:36:21.112325 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":89,"threads_started":1}
I20260811 14:36:21.112344 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:21.107134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:39187 (local address 127.0.0.1:58830)
0811 14:36:21.107258 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:21.107268 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:21.107350 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:21.107884 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:21.107885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:21.107887 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:36:21.107930 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:21.107931 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.110109 (+ 2178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.110110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:21.111137 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:21.111140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.112107 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.112108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:21.112171 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:21.112176 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:36:21.112234 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 14:36:21.112279 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":93,"threads_started":1}
I20260811 14:36:21.112752 19940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db473d335c0445829d9f27a985013587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:36:21.113198 20146 tablet_service.cc:1467] Processing CreateTablet for tablet db473d335c0445829d9f27a985013587 (DEFAULT_TABLE table=test-table [id=fcdce2a4dacf41d7ade690efd6ba88c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:36:21.113274 19713 tablet_service.cc:1467] Processing CreateTablet for tablet db473d335c0445829d9f27a985013587 (DEFAULT_TABLE table=test-table [id=fcdce2a4dacf41d7ade690efd6ba88c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:36:21.113461 20146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db473d335c0445829d9f27a985013587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:36:21.113610 19713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db473d335c0445829d9f27a985013587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:36:21.115998 20267 tablet_bootstrap.cc:492] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4: Bootstrap starting.
I20260811 14:36:21.116688 20267 tablet_bootstrap.cc:654] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4: Neither blocks nor log segments found. Creating new log.
I20260811 14:36:21.117061 20269 tablet_bootstrap.cc:492] T db473d335c0445829d9f27a985013587 P fd476598415740c0b44492d4be66575c: Bootstrap starting.
I20260811 14:36:21.117252 20267 log.cc:824] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4: Log is configured to *not* fsync() on all Append() calls
I20260811 14:36:21.117268 20268 tablet_bootstrap.cc:492] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365: Bootstrap starting.
I20260811 14:36:21.117743 20269 tablet_bootstrap.cc:654] T db473d335c0445829d9f27a985013587 P fd476598415740c0b44492d4be66575c: Neither blocks nor log segments found. Creating new log.
I20260811 14:36:21.117895 20268 tablet_bootstrap.cc:654] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365: Neither blocks nor log segments found. Creating new log.
I20260811 14:36:21.118055 20267 tablet_bootstrap.cc:492] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4: No bootstrap required, opened a new log
I20260811 14:36:21.118137 20267 ts_tablet_manager.cc:1397] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:36:21.118373 20269 log.cc:824] T db473d335c0445829d9f27a985013587 P fd476598415740c0b44492d4be66575c: Log is configured to *not* fsync() on all Append() calls
I20260811 14:36:21.118497 20268 log.cc:824] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365: Log is configured to *not* fsync() on all Append() calls
I20260811 14:36:21.119303 20268 tablet_bootstrap.cc:492] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365: No bootstrap required, opened a new log
I20260811 14:36:21.119361 20268 ts_tablet_manager.cc:1397] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:36:21.119939 20267 raft_consensus.cc:348] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbcd69d55374b72a02b0da00c2ec365" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 32821 } } peers { permanent_uuid: "e3f9596e08ca467daf1f763e6fb669a4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39979 } } peers { permanent_uuid: "fd476598415740c0b44492d4be66575c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 39187 } }
I20260811 14:36:21.120146 20267 raft_consensus.cc:374] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:36:21.120198 20267 raft_consensus.cc:729] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3f9596e08ca467daf1f763e6fb669a4, State: Initialized, Role: FOLLOWER
I20260811 14:36:21.120328 20267 consensus_queue.cc:260] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbcd69d55374b72a02b0da00c2ec365" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 32821 } } peers { permanent_uuid: "e3f9596e08ca467daf1f763e6fb669a4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39979 } } peers { permanent_uuid: "fd476598415740c0b44492d4be66575c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 39187 } }
I20260811 14:36:21.120652 20267 ts_tablet_manager.cc:1428] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 14:36:21.120671 20243 heartbeater.cc:503] Master 127.2.103.126:32901 was elected leader, sending a full tablet report...
I20260811 14:36:21.121308 20269 tablet_bootstrap.cc:492] T db473d335c0445829d9f27a985013587 P fd476598415740c0b44492d4be66575c: No bootstrap required, opened a new log
I20260811 14:36:21.121387 20269 ts_tablet_manager.cc:1397] T db473d335c0445829d9f27a985013587 P fd476598415740c0b44492d4be66575c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 14:36:21.121376 20268 raft_consensus.cc:348] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbcd69d55374b72a02b0da00c2ec365" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 32821 } } peers { permanent_uuid: "e3f9596e08ca467daf1f763e6fb669a4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39979 } } peers { permanent_uuid: "fd476598415740c0b44492d4be66575c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 39187 } }
I20260811 14:36:21.121559 20268 raft_consensus.cc:374] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:36:21.121603 20268 raft_consensus.cc:729] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbcd69d55374b72a02b0da00c2ec365, State: Initialized, Role: FOLLOWER
I20260811 14:36:21.121716 20268 consensus_queue.cc:260] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbcd69d55374b72a02b0da00c2ec365" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 32821 } } peers { permanent_uuid: "e3f9596e08ca467daf1f763e6fb669a4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39979 } } peers { permanent_uuid: "fd476598415740c0b44492d4be66575c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 39187 } }
I20260811 14:36:21.121984 20268 ts_tablet_manager.cc:1428] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:36:21.122138 20037 heartbeater.cc:503] Master 127.2.103.126:32901 was elected leader, sending a full tablet report...
I20260811 14:36:21.123481 20269 raft_consensus.cc:348] T db473d335c0445829d9f27a985013587 P fd476598415740c0b44492d4be66575c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbcd69d55374b72a02b0da00c2ec365" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 32821 } } peers { permanent_uuid: "e3f9596e08ca467daf1f763e6fb669a4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39979 } } peers { permanent_uuid: "fd476598415740c0b44492d4be66575c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 39187 } }
I20260811 14:36:21.123600 20269 raft_consensus.cc:374] T db473d335c0445829d9f27a985013587 P fd476598415740c0b44492d4be66575c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:36:21.123620 20269 raft_consensus.cc:729] T db473d335c0445829d9f27a985013587 P fd476598415740c0b44492d4be66575c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd476598415740c0b44492d4be66575c, State: Initialized, Role: FOLLOWER
I20260811 14:36:21.123724 20269 consensus_queue.cc:260] T db473d335c0445829d9f27a985013587 P fd476598415740c0b44492d4be66575c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbcd69d55374b72a02b0da00c2ec365" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 32821 } } peers { permanent_uuid: "e3f9596e08ca467daf1f763e6fb669a4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39979 } } peers { permanent_uuid: "fd476598415740c0b44492d4be66575c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 39187 } }
I20260811 14:36:21.124027 20269 ts_tablet_manager.cc:1428] T db473d335c0445829d9f27a985013587 P fd476598415740c0b44492d4be66575c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:36:21.124051 19824 heartbeater.cc:503] Master 127.2.103.126:32901 was elected leader, sending a full tablet report...
I20260811 14:36:21.130695 20274 raft_consensus.cc:482] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:36:21.130856 20274 raft_consensus.cc:504] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbcd69d55374b72a02b0da00c2ec365" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 32821 } } peers { permanent_uuid: "e3f9596e08ca467daf1f763e6fb669a4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39979 } } peers { permanent_uuid: "fd476598415740c0b44492d4be66575c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 39187 } }
I20260811 14:36:21.131278 20274 leader_election.cc:288] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bbcd69d55374b72a02b0da00c2ec365 (127.2.103.66:32821), fd476598415740c0b44492d4be66575c (127.2.103.65:39187)
I20260811 14:36:21.135114 20265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:21.131545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:55529 (local address 127.2.103.65:39187)
0811 14:36:21.131639 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:21.131642 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:21.131650 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:36:21.131704 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:21.131705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:21.131716 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:21.131816 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:36:21.132006 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.132763 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.134139 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.134760 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.134816 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:21.134824 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:21.135028 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 14:36:21.135060 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 14:36:21.135134 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:21.131432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:39187 (local address 127.2.103.67:55529)
0811 14:36:21.131523 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:21.131535 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:21.131580 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:21.131827 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:21.131828 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:21.131830 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:36:21.131911 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:21.131913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.132868 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.132869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:21.134044 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:21.134046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.134857 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.134858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:21.134948 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:21.134952 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:36:21.135017 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 14:36:21.135060 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 14:36:21.135617 20278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:21.131682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:32821 (local address 127.2.103.67:41053)
0811 14:36:21.131884 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:21.131893 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:36:21.131946 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:36:21.132324 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:36:21.132325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:36:21.132327 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:36:21.132413 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:21.132415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.133416 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.133417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:21.134519 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:36:21.134521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.135370 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.135371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:36:21.135448 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:21.135452 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:36:21.135500 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 14:36:21.135534 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":117,"threads_started":1}
I20260811 14:36:21.135622 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:21.131644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:41053 (local address 127.2.103.66:32821)
0811 14:36:21.131696 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:21.131699 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:21.132112 (+ 413us) server_negotiation.cc:407] Connection header received
0811 14:36:21.132154 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:21.132155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:21.132165 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:21.132252 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:36:21.132558 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.133322 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.134601 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.135176 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.135243 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:21.135251 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:21.135532 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 14:36:21.135576 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 14:36:21.135569 19743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db473d335c0445829d9f27a985013587" candidate_uuid: "e3f9596e08ca467daf1f763e6fb669a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd476598415740c0b44492d4be66575c" is_pre_election: true
I20260811 14:36:21.135743 19743 raft_consensus.cc:2454] T db473d335c0445829d9f27a985013587 P fd476598415740c0b44492d4be66575c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3f9596e08ca467daf1f763e6fb669a4 in term 0.
I20260811 14:36:21.135993 20080 leader_election.cc:302] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e3f9596e08ca467daf1f763e6fb669a4, fd476598415740c0b44492d4be66575c; no voters:
I20260811 14:36:21.136024 19977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db473d335c0445829d9f27a985013587" candidate_uuid: "e3f9596e08ca467daf1f763e6fb669a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbcd69d55374b72a02b0da00c2ec365" is_pre_election: true
I20260811 14:36:21.136125 20274 raft_consensus.cc:2781] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:36:21.136185 20274 raft_consensus.cc:482] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:36:21.136180 19977 raft_consensus.cc:2454] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3f9596e08ca467daf1f763e6fb669a4 in term 0.
I20260811 14:36:21.136219 20274 raft_consensus.cc:3037] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:36:21.137059 20274 raft_consensus.cc:504] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbcd69d55374b72a02b0da00c2ec365" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 32821 } } peers { permanent_uuid: "e3f9596e08ca467daf1f763e6fb669a4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39979 } } peers { permanent_uuid: "fd476598415740c0b44492d4be66575c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 39187 } }
I20260811 14:36:21.137203 20274 leader_election.cc:288] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [CANDIDATE]: Term 1 election: Requested vote from peers 6bbcd69d55374b72a02b0da00c2ec365 (127.2.103.66:32821), fd476598415740c0b44492d4be66575c (127.2.103.65:39187)
I20260811 14:36:21.137377 19977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db473d335c0445829d9f27a985013587" candidate_uuid: "e3f9596e08ca467daf1f763e6fb669a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbcd69d55374b72a02b0da00c2ec365"
I20260811 14:36:21.137454 19977 raft_consensus.cc:3037] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:36:21.137458 19743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db473d335c0445829d9f27a985013587" candidate_uuid: "e3f9596e08ca467daf1f763e6fb669a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd476598415740c0b44492d4be66575c"
I20260811 14:36:21.137529 19743 raft_consensus.cc:3037] T db473d335c0445829d9f27a985013587 P fd476598415740c0b44492d4be66575c [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:36:21.138221 19743 raft_consensus.cc:2454] T db473d335c0445829d9f27a985013587 P fd476598415740c0b44492d4be66575c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3f9596e08ca467daf1f763e6fb669a4 in term 1.
I20260811 14:36:21.138298 19977 raft_consensus.cc:2454] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3f9596e08ca467daf1f763e6fb669a4 in term 1.
I20260811 14:36:21.138386 20080 leader_election.cc:302] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e3f9596e08ca467daf1f763e6fb669a4, fd476598415740c0b44492d4be66575c; no voters:
I20260811 14:36:21.138502 20274 raft_consensus.cc:2781] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:36:21.138707 20274 raft_consensus.cc:686] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [term 1 LEADER]: Becoming Leader. State: Replica: e3f9596e08ca467daf1f763e6fb669a4, State: Running, Role: LEADER
I20260811 14:36:21.138816 20274 consensus_queue.cc:237] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbcd69d55374b72a02b0da00c2ec365" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 32821 } } peers { permanent_uuid: "e3f9596e08ca467daf1f763e6fb669a4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39979 } } peers { permanent_uuid: "fd476598415740c0b44492d4be66575c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 39187 } }
I20260811 14:36:21.139678 19507 catalog_manager.cc:5495] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3f9596e08ca467daf1f763e6fb669a4 (127.2.103.67). New cstate: current_term: 1 leader_uuid: "e3f9596e08ca467daf1f763e6fb669a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbcd69d55374b72a02b0da00c2ec365" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 32821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3f9596e08ca467daf1f763e6fb669a4" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd476598415740c0b44492d4be66575c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 39187 } health_report { overall_health: UNKNOWN } } }
W20260811 14:36:21.146042 20038 tablet.cc:2369] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:36:21.148780 20263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:21.146377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38458 (local address 127.2.103.67:39979)
0811 14:36:21.146431 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 14:36:21.146433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:36:21.146443 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:36:21.146519 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:36:21.146520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:36:21.146546 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:36:21.146651 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:36:21.146756 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.147220 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.147991 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:36:21.148180 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:36:21.148257 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:36:21.148423 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:36:21.148525 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:36:21.148613 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 14:36:21.148655 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 14:36:21.148677 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 14:36:21.150550 19977 raft_consensus.cc:1261] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365 [term 1 FOLLOWER]: Refusing update from remote peer e3f9596e08ca467daf1f763e6fb669a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:36:21.150575 19743 raft_consensus.cc:1261] T db473d335c0445829d9f27a985013587 P fd476598415740c0b44492d4be66575c [term 1 FOLLOWER]: Refusing update from remote peer e3f9596e08ca467daf1f763e6fb669a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:36:21.151041 20279 consensus_queue.cc:1035] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd476598415740c0b44492d4be66575c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 39187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:36:21.151172 20274 consensus_queue.cc:1035] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbcd69d55374b72a02b0da00c2ec365" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 32821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:36:21.154191 20287 mvcc.cc:205] Tried to move back new op lower bound from 7317335986790133760 to 7317335986745466880. Current Snapshot: MvccSnapshot[applied={T|T < 7317335986790133760}]
I20260811 14:36:21.156489 19506 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:36:21.156579 19506 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:36:21.157361 19506 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 953eff2253334f64a8446730488b0029: Sending DeleteTablet for 3 replicas of tablet db473d335c0445829d9f27a985013587
I20260811 14:36:21.157723 20146 tablet_service.cc:1514] Processing DeleteTablet for tablet db473d335c0445829d9f27a985013587 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:36:21 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:38456
I20260811 14:36:21.157707 19940 tablet_service.cc:1514] Processing DeleteTablet for tablet db473d335c0445829d9f27a985013587 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:36:21 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:43546
I20260811 14:36:21.158025 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 19614
I20260811 14:36:21.158016 19713 tablet_service.cc:1514] Processing DeleteTablet for tablet db473d335c0445829d9f27a985013587 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:36:21 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:58830
I20260811 14:36:21.158641 20303 tablet_replica.cc:330] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4: stopping tablet replica
I20260811 14:36:21.158762 20303 raft_consensus.cc:2229] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:36:21.158934 20303 raft_consensus.cc:2258] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:36:21.161396 20303 ts_tablet_manager.cc:1905] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:36:21.162004 20309 tablet_replica.cc:330] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365: stopping tablet replica
I20260811 14:36:21.162201 20309 raft_consensus.cc:2229] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:36:21.162405 20309 raft_consensus.cc:2258] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:36:21.163048 20309 ts_tablet_manager.cc:1905] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:36:21.163664 20303 ts_tablet_manager.cc:1918] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:36:21.163767 20303 log.cc:1196] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/db473d335c0445829d9f27a985013587
I20260811 14:36:21.164132 20303 ts_tablet_manager.cc:1939] T db473d335c0445829d9f27a985013587 P e3f9596e08ca467daf1f763e6fb669a4: Deleting consensus metadata
I20260811 14:36:21.164721 19486 catalog_manager.cc:4841] TS e3f9596e08ca467daf1f763e6fb669a4 (127.2.103.67:39979): tablet db473d335c0445829d9f27a985013587 (table test-table [id=fcdce2a4dacf41d7ade690efd6ba88c4]) successfully deleted
I20260811 14:36:21.165326 20309 ts_tablet_manager.cc:1918] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:36:21.165494 20309 log.cc:1196] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/db473d335c0445829d9f27a985013587
I20260811 14:36:21.165925 20309 ts_tablet_manager.cc:1939] T db473d335c0445829d9f27a985013587 P 6bbcd69d55374b72a02b0da00c2ec365: Deleting consensus metadata
I20260811 14:36:21.166510 19484 catalog_manager.cc:4841] TS 6bbcd69d55374b72a02b0da00c2ec365 (127.2.103.66:32821): tablet db473d335c0445829d9f27a985013587 (table test-table [id=fcdce2a4dacf41d7ade690efd6ba88c4]) successfully deleted
W20260811 14:36:21.174991 19486 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.103.65:39187 (error 108)
I20260811 14:36:21.175745 20257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:36:21.175563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:39187 (local address 127.0.0.1:58844)
0811 14:36:21.175635 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0811 14:36:21.175691 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.65:39187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 14:36:21.175819 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 19831
W20260811 14:36:21.184072 19486 catalog_manager.cc:4584] TS fd476598415740c0b44492d4be66575c (127.2.103.65:39187): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet db473d335c0445829d9f27a985013587: Network error: Client connection negotiation failed: client connection to 127.2.103.65:39187: connect: Connection refused (error 111)
I20260811 14:36:21.190533 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 20054
I20260811 14:36:21.202991 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 19457
2026-08-11T14:36:21Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2263 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:36:21 dist-test-slave-twwt krb5kdc[20314](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:36:21 dist-test-slave-twwt krb5kdc[20314](info): set up 2 sockets
Aug 11 14:36:21 dist-test-slave-twwt krb5kdc[20314](info): commencing operation
krb5kdc: starting...
W20260811 14:36:22.481169 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:36:22 dist-test-slave-twwt krb5kdc[20314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:36:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:36:22Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.2.103.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.2.103.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.2.103.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/rangeradmin_127.2.103.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.2.103.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/rangeradmin_127.2.103.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.2.103.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.2.103.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.2.103.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/rangerlookup_127.2.103.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.2.103.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/rangerlookup_127.2.103.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.103.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.103.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.103.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/HTTP_127.2.103.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.103.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/HTTP_127.2.103.84@KRBTEST.COM.keytab.
I20260811 14:36:22.666502 2461 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskmCSHXK/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/postgres -l logfile start
2026-08-11 14:36:25.350 UTC [20358] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 14:36:25.351 UTC [20358] LOG: listening on IPv4 address "127.2.103.84", port 41505
2026-08-11 14:36:25.355 UTC [20358] LOG: listening on Unix socket "/tmp/.s.PGSQL.41505"
2026-08-11 14:36:25.377 UTC [20363] LOG: database system was shut down at 2026-08-11 14:36:23 UTC
2026-08-11 14:36:25.382 UTC [20358] LOG: database system is ready to accept connections
I20260811 14:36:26.637253 2461 mini_postgres.cc:96] Postgres bound to 41505
2026-08-11 14:36:26.645 UTC [20375] FATAL: database "slave" does not exist
127.2.103.84:41505 - accepting connections
I20260811 14:36:26.646356 2461 mini_ranger.cc:161] Starting Ranger...
I20260811 14:36:26.663201 2461 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 14:36:27.174304 2461 mini_ranger.cc:87] Created ranger Postgres database
I20260811 14:36:27.174407 2461 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ranger-admin
2026-08-11 14:36:27,556 [I] DB FLAVOR :POSTGRES
2026-08-11 14:36:27,556 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 14:36:27,557 [I] Checking connection..
2026-08-11 14:36:27,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 14:36:27,850 [I] Checking connection passed.
2026-08-11 14:36:27,850 [I] --------- Verifying version history table ---------
2026-08-11 14:36:27,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 14:36:28,131 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 14:36:28,131 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 14:36:28,387 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 14:36:28,388 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 14:36:28,388 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 14:36:28.658 UTC [20467] WARNING: there is no transaction in progress
2026-08-11 14:36:28,665 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 14:36:28,665 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 14:36:28,935 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 14:36:28,935 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 14:36:28,935 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 14:36:29,186 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 14:36:29,458 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-twwt.c.gcp-upstream.internal','N') ;"
2026-08-11 14:36:29,722 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 14:36:29,983 [I] Table x_portal_user does not exist in database ranger
2026-08-11 14:36:29,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 14:36:30,257 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 14:36:30,257 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 14:36:30,525 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 14:36:30,784 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 14:36:30,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 14:36:31.804 UTC [20632] WARNING: there is no transaction in progress
2026-08-11 14:36:31.819 UTC [20632] WARNING: there is no transaction in progress
2026-08-11 14:36:31.837 UTC [20632] WARNING: there is no transaction in progress
2026-08-11 14:36:31.853 UTC [20632] WARNING: there is no transaction in progress
2026-08-11 14:36:31.869 UTC [20632] WARNING: there is no transaction in progress
2026-08-11 14:36:31.891 UTC [20632] WARNING: there is no transaction in progress
2026-08-11 14:36:31.948 UTC [20632] WARNING: there is no transaction in progress
2026-08-11 14:36:31.956 UTC [20632] WARNING: there is no transaction in progress
2026-08-11 14:36:31.973 UTC [20632] WARNING: there is no transaction in progress
2026-08-11 14:36:31.984 UTC [20632] WARNING: there is no transaction in progress
2026-08-11 14:36:31.995 UTC [20632] WARNING: there is no transaction in progress
2026-08-11 14:36:32.009 UTC [20632] WARNING: there is no transaction in progress
2026-08-11 14:36:32.020 UTC [20632] WARNING: there is no transaction in progress
2026-08-11 14:36:32.026 UTC [20632] WARNING: there is no transaction in progress
2026-08-11 14:36:32.034 UTC [20632] WARNING: there is no transaction in progress
2026-08-11 14:36:32.042 UTC [20632] WARNING: there is no transaction in progress
2026-08-11 14:36:32,750 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 14:36:32,750 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 14:36:32,998 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 14:36:32,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 14:36:33,274 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 14:36:33,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 14:36:33,542 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 14:36:33,542 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 14:36:33,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 14:36:34,077 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-twwt.c.gcp-upstream.internal';"
2026-08-11 14:36:34,336 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 14:36:34,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 14:36:34,597 [I] DB_PATCHES have already been applied
I20260811 14:36:34.603173 2461 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ranger-admin:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskmCSHXK/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 14:36:34.603255 2461 mini_ranger.cc:193] Using host: 127.2.103.84
I20260811 14:36:34.607431 2461 mini_ranger.cc:227] Ranger admin URL: http://127.2.103.84:59163
Aug 11, 2026 2:36:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 2:36:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 2:36:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 2:36:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 2:36:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 2:36:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 2:36:35 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 14:36:36.052686 2461 mini_ranger.cc:160] Time spent starting Ranger: real 9.406s user 0.005s sys 0.007s
Aug 11, 2026 2:36:57 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 2:36:57 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 2:36:57 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 2:36:57 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 2:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 2:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 2:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 2:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 2:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 2:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 2:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 2:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 2:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 2:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 2:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 2:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 2:36:57 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 2:36:58 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 14:36:59.008957 2461 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.2.103.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.2.103.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.2.103.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/rangerkms_127.2.103.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.2.103.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/rangerkms_127.2.103.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.103.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.103.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.103.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/HTTP_127.2.103.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.103.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/HTTP_127.2.103.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 14:36:59 dist-test-slave-twwt krb5kdc[20314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459019, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 14:36:59.102592 2461 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 14:36:59.114216 2461 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 14:37:00.113837 2461 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 14:37:00.113915 2461 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ranger-kms
I20260811 14:37:00.113929 2461 mini_ranger_kms.cc:227] Using postgres at 127.2.103.84:41505
2026-08-11 14:37:00,423 [I] DB FLAVOR :POSTGRES
2026-08-11 14:37:00,423 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 14:37:00,423 [I] Checking connection
2026-08-11 14:37:00,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 14:37:00,709 [I] connection success
2026-08-11 14:37:00,709 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 14:37:00,709 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 14:37:00,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 14:37:01,016 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 14:37:01,017 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 14:37:01,017 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 14:37:01,017 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:41505/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 14:37:01,307 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 14:37:01.876276 2461 mini_ranger_kms.cc:310] Created kms service
I20260811 14:37:02.074290 2461 mini_ranger_kms.cc:326] Created kudu user
I20260811 14:37:02.120595 2461 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 14:37:02 dist-test-slave-twwt krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459019, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
I20260811 14:37:02.557359 2461 mini_ranger_kms.cc:384] Added ranger policy
I20260811 14:37:02.557487 2461 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ranger-kms:/tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 14:37:02.557509 2461 mini_ranger_kms.cc:240] Using host: 127.2.103.84
I20260811 14:37:02.562927 2461 mini_ranger_kms.cc:276] Ranger KMS PID: 20931
I20260811 14:37:02.563069 2461 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.2.103.84:58257
Aug 11, 2026 2:37:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 2:37:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 2:37:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 2:37:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 2:37:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 2:37:04 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 14:37:04 dist-test-slave-twwt krb5kdc[20314](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786459024, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.103.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:37:05.351078 2461 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.248s user 0.002s sys 0.288s
I20260811 14:37:05.351189 2461 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 14:37:05.351266 2461 mini_ranger_kms.cc:401] 127.2.103.84:58257/kms/v1/keys
Aug 11 14:37:05 dist-test-slave-twwt krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459024, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.103.84@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
Aug 11 14:37:05 dist-test-slave-twwt krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459024, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.103.84@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
Aug 11 14:37:08 dist-test-slave-twwt krb5kdc[20314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:37:08.290040 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:41353
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:39387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:41353
--ranger_config_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.103.84:58257/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:37:08.398577 21005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:37:08.398886 21005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:37:08.398912 21005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:37:08.400123 21005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:37:08.400171 21005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:37:08.400185 21005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:37:08.400198 21005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:37:08.400255 21005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:37:08.401948 21005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:39387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/master-0/wal
--ranger_kms_url=127.2.103.84:58257/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:41353
--ranger_config_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:41353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:37:08.402220 21005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:37:08.402491 21005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:37:08.407928 21016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:37:08.408008 21015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:37:08.407939 21018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:37:08.408130 21005 server_base.cc:1034] running on GCE node
I20260811 14:37:08.408502 21005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:37:08.408866 21005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:37:08.410034 21005 hybrid_clock.cc:648] HybridClock initialized: now 1786459028410018 us; error 48 us; skew 500 ppm
Aug 11 14:37:08 dist-test-slave-twwt krb5kdc[20314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459028, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:37:08.412971 21005 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:37:08.413513 21005 webserver.cc:460] Webserver started at http://127.2.103.126:40597/ using document root <none> and password file <none>
I20260811 14:37:08.413782 21005 fs_manager.cc:359] Metadata directory not provided
I20260811 14:37:08.413831 21005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:37:08.414011 21005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 14:37:08 dist-test-slave-twwt krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459028, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
I20260811 14:37:08.469153 21005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "f3cf4bca4fd54c8dbd44d69dee771ef5"
format_stamp: "Formatted at 2026-08-11 14:37:08 on dist-test-slave-twwt"
server_key: "0bd87e1d75e17f9f4a5d3a53592080bf"
server_key_iv: "4e2ee4065f2881c36c5d780ac0ebcdd0"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:08.469641 21005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "f3cf4bca4fd54c8dbd44d69dee771ef5"
format_stamp: "Formatted at 2026-08-11 14:37:08 on dist-test-slave-twwt"
server_key: "0bd87e1d75e17f9f4a5d3a53592080bf"
server_key_iv: "4e2ee4065f2881c36c5d780ac0ebcdd0"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:08.471958 21005 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260811 14:37:08.485452 21027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:08.485731 21005 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:37:08.485862 21005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "f3cf4bca4fd54c8dbd44d69dee771ef5"
format_stamp: "Formatted at 2026-08-11 14:37:08 on dist-test-slave-twwt"
server_key: "0bd87e1d75e17f9f4a5d3a53592080bf"
server_key_iv: "4e2ee4065f2881c36c5d780ac0ebcdd0"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:08.486001 21005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:37:08.509835 21005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:37:08.511966 21005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:37:08.512176 21005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:37:08.517792 21005 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:41353
I20260811 14:37:08.517861 21109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:41353 every 8 connection(s)
I20260811 14:37:08.518365 21005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:37:08.519182 21111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:37:08.521658 21111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5: Bootstrap starting.
I20260811 14:37:08.522305 21111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5: Neither blocks nor log segments found. Creating new log.
I20260811 14:37:08.522681 21111 log.cc:824] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5: Log is configured to *not* fsync() on all Append() calls
I20260811 14:37:08.523527 21111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5: No bootstrap required, opened a new log
I20260811 14:37:08.525787 21111 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cf4bca4fd54c8dbd44d69dee771ef5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41353 } }
I20260811 14:37:08.525959 21111 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:37:08.525982 21111 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3cf4bca4fd54c8dbd44d69dee771ef5, State: Initialized, Role: FOLLOWER
I20260811 14:37:08.526156 21111 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cf4bca4fd54c8dbd44d69dee771ef5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41353 } }
I20260811 14:37:08.526237 21111 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:37:08.526279 21111 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:37:08.526336 21111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:37:08.526968 21111 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cf4bca4fd54c8dbd44d69dee771ef5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41353 } }
I20260811 14:37:08.527086 21111 leader_election.cc:302] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3cf4bca4fd54c8dbd44d69dee771ef5; no voters:
I20260811 14:37:08.527271 21111 leader_election.cc:288] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:37:08.527328 21115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:37:08.527550 21115 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5 [term 1 LEADER]: Becoming Leader. State: Replica: f3cf4bca4fd54c8dbd44d69dee771ef5, State: Running, Role: LEADER
I20260811 14:37:08.527571 21111 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:37:08.527653 21115 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cf4bca4fd54c8dbd44d69dee771ef5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41353 } }
I20260811 14:37:08.528242 21120 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3cf4bca4fd54c8dbd44d69dee771ef5. Latest consensus state: current_term: 1 leader_uuid: "f3cf4bca4fd54c8dbd44d69dee771ef5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cf4bca4fd54c8dbd44d69dee771ef5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41353 } } }
I20260811 14:37:08.528384 21120 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:37:08.528673 21133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:37:08.528887 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 21005
I20260811 14:37:08.529019 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:37:08.529241 2461 external_mini_cluster.cc:1428] Setting key 21f254375fcb55b560771079730aaa95
I20260811 14:37:08.528267 21119 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3cf4bca4fd54c8dbd44d69dee771ef5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cf4bca4fd54c8dbd44d69dee771ef5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41353 } } }
I20260811 14:37:08.529299 21133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:37:08.529354 21119 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:37:08.529855 21111 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 14:37:08.535655 21133 catalog_manager.cc:1347] Generated new cluster ID: c480b73694e947dd8d0dae742a361739
I20260811 14:37:08.535727 21133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:37:08 dist-test-slave-twwt krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:37:08.544451 21152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:08.530009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32990 (local address 127.2.103.126:41353)
0811 14:37:08.530494 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 14:37:08.530506 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 14:37:08.530542 (+ 36us) server_negotiation.cc:407] Connection header received
0811 14:37:08.532349 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:37:08.532399 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:37:08.532618 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:37:08.532882 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:37:08.534177 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:08.535877 (+ 1700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:08.536968 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:08.537596 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:08.537997 (+ 401us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:08.540553 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:37:08.540557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:37:08.540557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:37:08.540570 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:37:08.542566 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:08.542977 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:08.542980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:08.542980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:08.543088 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:08.543261 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:08.543261 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:08.543262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:08.543554 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:37:08.544024 (+ 470us) server_negotiation.cc:1035] Waiting for connection context
0811 14:37:08.544093 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 14:37:08.544207 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":139,"threads_started":1}
I20260811 14:37:08.568585 21133 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:37:08.569172 21133 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:37:08.575718 21133 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5: Generated new TSK 0
I20260811 14:37:08.575941 21133 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
I20260811 14:37:09.357932 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.103.84:58257/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41353
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:39387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:37:09.477967 21196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:37:09.478204 21196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:37:09.478245 21196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:37:09.479476 21196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:37:09.479543 21196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:37:09.479591 21196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:37:09.481334 21196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:39387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.2.103.84:58257/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:37:09.481588 21196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:37:09.481871 21196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:37:09.488466 21214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:37:09.488667 21205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:37:09.488638 21196 server_base.cc:1034] running on GCE node
W20260811 14:37:09.488489 21208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:37:09.489039 21196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:37:09.489319 21196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:37:09.490517 21196 hybrid_clock.cc:648] HybridClock initialized: now 1786459029490488 us; error 43 us; skew 500 ppm
Aug 11 14:37:09 dist-test-slave-twwt krb5kdc[20314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459029, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:37:09.494228 21196 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:37:09.494869 21196 webserver.cc:460] Webserver started at http://127.2.103.65:33693/ using document root <none> and password file <none>
I20260811 14:37:09.495144 21196 fs_manager.cc:359] Metadata directory not provided
I20260811 14:37:09.495213 21196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:37:09.495371 21196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 14:37:09 dist-test-slave-twwt krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459029, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
I20260811 14:37:09.510391 21196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "ffe511f326084fd8a38e61fedc1f6217"
format_stamp: "Formatted at 2026-08-11 14:37:09 on dist-test-slave-twwt"
server_key: "cedf7b96b67731cdbb4abfb9f5ca75af"
server_key_iv: "978caf1b9c7f56ce1d1c125e1d7f036f"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:09.511015 21196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "ffe511f326084fd8a38e61fedc1f6217"
format_stamp: "Formatted at 2026-08-11 14:37:09 on dist-test-slave-twwt"
server_key: "cedf7b96b67731cdbb4abfb9f5ca75af"
server_key_iv: "978caf1b9c7f56ce1d1c125e1d7f036f"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:09.513222 21196 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 14:37:09.530256 21223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:09.530578 21196 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:37:09.530774 21196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "ffe511f326084fd8a38e61fedc1f6217"
format_stamp: "Formatted at 2026-08-11 14:37:09 on dist-test-slave-twwt"
server_key: "cedf7b96b67731cdbb4abfb9f5ca75af"
server_key_iv: "978caf1b9c7f56ce1d1c125e1d7f036f"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:09.530891 21196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:37:09.545217 21196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:37:09.547706 21196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:37:09.547885 21196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:37:09.548202 21196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:37:09.548605 21196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:37:09.548661 21196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:09.548731 21196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:37:09.548777 21196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:09.558182 21196 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:35005
I20260811 14:37:09.558670 21196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:37:09.558802 21401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:35005 every 8 connection(s)
I20260811 14:37:09.559410 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 21196
I20260811 14:37:09.559540 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:37:09.559731 2461 external_mini_cluster.cc:1428] Setting key e4f551bc9c5d1be791609593dfe05f85
Aug 11 14:37:09 dist-test-slave-twwt krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459029, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.103.66@KRBTEST.COM; defaulting to no policy
I20260811 14:37:09.576602 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:09.560414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:59671 (local address 127.2.103.126:41353)
0811 14:37:09.560858 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 14:37:09.560860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:37:09.561101 (+ 241us) server_negotiation.cc:407] Connection header received
0811 14:37:09.561908 (+ 807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:37:09.561909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:37:09.561925 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:37:09.561977 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:37:09.566972 (+ 4995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:09.567766 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:09.569416 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:09.569734 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:09.569785 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:09.572644 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:37:09.572646 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:37:09.572646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:37:09.572654 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:37:09.574821 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:09.575360 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:09.575362 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:09.575362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:09.575429 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:09.575724 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:09.575725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:09.575725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:09.575886 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:37:09.575939 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 14:37:09.576387 (+ 448us) server_negotiation.cc:299] Negotiation successful
0811 14:37:09.576484 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":106,"threads_started":1}
I20260811 14:37:09.577143 21404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:09.560389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:41353 (local address 127.2.103.65:59671)
0811 14:37:09.560945 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 14:37:09.560998 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 14:37:09.561714 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:37:09.562197 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:37:09.562201 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:37:09.566401 (+ 4200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:37:09.566812 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:09.566824 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:09.567921 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:09.567923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:09.569297 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:09.569301 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:09.569902 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:09.569903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:09.570064 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:09.570738 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:37:09.570750 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:37:09.572517 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:37:09.574934 (+ 2417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:37:09.574942 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:37:09.574945 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:37:09.575261 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:37:09.575503 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:37:09.575504 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:37:09.575504 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:37:09.575644 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:37:09.575993 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:37:09.576001 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:37:09.576175 (+ 174us) client_negotiation.cc:769] Sending connection context
0811 14:37:09.576712 (+ 537us) client_negotiation.cc:240] Negotiation successful
0811 14:37:09.576903 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":109,"threads_started":1}
I20260811 14:37:09.577814 21402 heartbeater.cc:347] Connected to a master server at 127.2.103.126:41353
I20260811 14:37:09.577934 21402 heartbeater.cc:464] Registering TS with master...
I20260811 14:37:09.578233 21402 heartbeater.cc:511] Master 127.2.103.126:41353 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.66@KRBTEST.COM" created.
I20260811 14:37:09.578826 21055 ts_manager.cc:194] Registered new tserver with Master: ffe511f326084fd8a38e61fedc1f6217 (127.2.103.65:35005)
I20260811 14:37:09.579727 21055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:59671
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
I20260811 14:37:09.599663 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.103.84:58257/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41353
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:39387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 14:37:09 dist-test-slave-twwt krb5kdc[20314](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786459029, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 14:37:09.731875 21408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:37:09.732240 21408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:37:09.732332 21408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:37:09.734135 21408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:37:09.734254 21408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:37:09.734342 21408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:37:09.736655 21408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:39387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.2.103.84:58257/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:37:09.737125 21408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:37:09.737483 21408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:37:09.744388 21423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:37:09.744632 21417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:37:09.744745 21418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:37:09.745232 21408 server_base.cc:1034] running on GCE node
I20260811 14:37:09.745461 21408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:37:09.745739 21408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:37:09.753150 21408 hybrid_clock.cc:648] HybridClock initialized: now 1786459029753102 us; error 66 us; skew 500 ppm
Aug 11 14:37:09 dist-test-slave-twwt krb5kdc[20314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459029, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:37:09.756950 21408 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:37:09.757540 21408 webserver.cc:460] Webserver started at http://127.2.103.66:43329/ using document root <none> and password file <none>
I20260811 14:37:09.757858 21408 fs_manager.cc:359] Metadata directory not provided
I20260811 14:37:09.757929 21408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:37:09.758071 21408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 14:37:09 dist-test-slave-twwt krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459029, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
I20260811 14:37:09.780663 21408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "dad061a82a75441485384395e114037a"
format_stamp: "Formatted at 2026-08-11 14:37:09 on dist-test-slave-twwt"
server_key: "3379d250ea0cb74980f8b4d24e5bad02"
server_key_iv: "c8f9f61636c9b2f622a9eb60c675eae7"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:09.781369 21408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "dad061a82a75441485384395e114037a"
format_stamp: "Formatted at 2026-08-11 14:37:09 on dist-test-slave-twwt"
server_key: "3379d250ea0cb74980f8b4d24e5bad02"
server_key_iv: "c8f9f61636c9b2f622a9eb60c675eae7"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:09.783375 21408 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 14:37:09.798290 21437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:09.798492 21408 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:37:09.798591 21408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "dad061a82a75441485384395e114037a"
format_stamp: "Formatted at 2026-08-11 14:37:09 on dist-test-slave-twwt"
server_key: "3379d250ea0cb74980f8b4d24e5bad02"
server_key_iv: "c8f9f61636c9b2f622a9eb60c675eae7"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:09.798736 21408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:37:09.819056 21408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:37:09.821803 21408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:37:09.822006 21408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:37:09.822324 21408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:37:09.822808 21408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:37:09.822862 21408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:09.822906 21408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:37:09.822981 21408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:09.833006 21408 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:45435
I20260811 14:37:09.833588 21408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:37:09.834429 21607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:45435 every 8 connection(s)
I20260811 14:37:09.841627 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 21408
I20260811 14:37:09.841791 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:37:09.842001 2461 external_mini_cluster.cc:1428] Setting key 1953f87ac0269d63aad29ef864718728
Aug 11 14:37:09 dist-test-slave-twwt krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459029, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:37:09.853549 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:09.834913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:36561 (local address 127.2.103.126:41353)
0811 14:37:09.834974 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 14:37:09.834976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:37:09.838529 (+ 3553us) server_negotiation.cc:407] Connection header received
0811 14:37:09.840713 (+ 2184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:37:09.840716 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:37:09.840737 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:37:09.840830 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:37:09.843387 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:09.844176 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:09.845792 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:09.846131 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:09.846185 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:09.849596 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:37:09.849598 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:37:09.849598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:37:09.849607 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:37:09.851581 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:09.852231 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:09.852232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:09.852233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:09.852296 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:09.852632 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:09.852633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:09.852634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:09.852816 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:37:09.852908 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 14:37:09.853330 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 14:37:09.853443 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 14:37:09.854219 21611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:09.834507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:41353 (local address 127.2.103.66:36561)
0811 14:37:09.838293 (+ 3786us) negotiation.cc:107] Waiting for socket to connect
0811 14:37:09.838365 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 14:37:09.840519 (+ 2154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:37:09.840933 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:37:09.840939 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:37:09.841283 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:37:09.843244 (+ 1961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:09.843255 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:09.844316 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:09.844317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:09.845676 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:09.845679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:09.846354 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:09.846355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:09.846523 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:09.847476 (+ 953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:37:09.847488 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:37:09.849453 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:37:09.851720 (+ 2267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:37:09.851730 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:37:09.851734 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:37:09.852131 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:37:09.852368 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:37:09.852369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:37:09.852370 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:37:09.852534 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:37:09.852949 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:37:09.852958 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:37:09.853150 (+ 192us) client_negotiation.cc:769] Sending connection context
0811 14:37:09.853719 (+ 569us) client_negotiation.cc:240] Negotiation successful
0811 14:37:09.853934 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3711,"thread_start_us":137,"threads_started":1}
I20260811 14:37:09.855049 21608 heartbeater.cc:347] Connected to a master server at 127.2.103.126:41353
I20260811 14:37:09.855176 21608 heartbeater.cc:464] Registering TS with master...
I20260811 14:37:09.855423 21608 heartbeater.cc:511] Master 127.2.103.126:41353 requested a full tablet report, sending...
I20260811 14:37:09.856033 21055 ts_manager.cc:194] Registered new tserver with Master: dad061a82a75441485384395e114037a (127.2.103.66:45435)
I20260811 14:37:09.856598 21055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:36561
WARNING: no policy specified for kudu/127.2.103.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
I20260811 14:37:09.886104 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.103.84:58257/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41353
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:39387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:37:10.021984 21617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:37:10.022321 21617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:37:10.022362 21617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:37:10.024309 21617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:37:10.024402 21617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:37:10.024451 21617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:37:10.026983 21617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:39387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.2.103.84:58257/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:37:10.027350 21617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:37:10.027706 21617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:37:10.034716 21630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:37:10.034716 21629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:37:10.034673 21634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:37:10.035102 21617 server_base.cc:1034] running on GCE node
I20260811 14:37:10.035598 21617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:37:10.036175 21617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:37:10.037449 21617 hybrid_clock.cc:648] HybridClock initialized: now 1786459030037424 us; error 55 us; skew 500 ppm
Aug 11 14:37:10 dist-test-slave-twwt krb5kdc[20314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459030, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:37:10.042316 21617 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:37:10.042935 21617 webserver.cc:460] Webserver started at http://127.2.103.67:38863/ using document root <none> and password file <none>
I20260811 14:37:10.043211 21617 fs_manager.cc:359] Metadata directory not provided
I20260811 14:37:10.043291 21617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:37:10.043452 21617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 14:37:10 dist-test-slave-twwt krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459030, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
I20260811 14:37:10.059830 21617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "1ab6554aadad40c6a8a4a8f5e30e8814"
format_stamp: "Formatted at 2026-08-11 14:37:10 on dist-test-slave-twwt"
server_key: "f455922109779227b01d51f220452db4"
server_key_iv: "ec3f27081636607b538b02f874f62caf"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:10.060426 21617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "1ab6554aadad40c6a8a4a8f5e30e8814"
format_stamp: "Formatted at 2026-08-11 14:37:10 on dist-test-slave-twwt"
server_key: "f455922109779227b01d51f220452db4"
server_key_iv: "ec3f27081636607b538b02f874f62caf"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:10.071432 21617 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.002s sys 0.000s
I20260811 14:37:10.087060 21641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:10.087249 21617 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:37:10.087348 21617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "1ab6554aadad40c6a8a4a8f5e30e8814"
format_stamp: "Formatted at 2026-08-11 14:37:10 on dist-test-slave-twwt"
server_key: "f455922109779227b01d51f220452db4"
server_key_iv: "ec3f27081636607b538b02f874f62caf"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:10.087455 21617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:37:10.111245 21617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:37:10.114427 21617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:37:10.114668 21617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:37:10.114918 21617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:37:10.115356 21617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:37:10.115415 21617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:10.115458 21617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:37:10.115484 21617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:10.127244 21617 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:37959
I20260811 14:37:10.127805 21617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:37:10.130132 21820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:37959 every 8 connection(s)
I20260811 14:37:10.135241 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 21617
I20260811 14:37:10.135360 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:37:10.135543 2461 external_mini_cluster.cc:1428] Setting key de7fb80b235db80d9a377bd80a6f079e
Aug 11 14:37:10 dist-test-slave-twwt krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459030, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:37:10.148663 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:10.130998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:40215 (local address 127.2.103.126:41353)
0811 14:37:10.131051 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 14:37:10.131054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:37:10.134139 (+ 3085us) server_negotiation.cc:407] Connection header received
0811 14:37:10.134971 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:37:10.134972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:37:10.134987 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:37:10.135022 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:37:10.136018 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.136804 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.138315 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.138657 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.138733 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:10.145108 (+ 6375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:37:10.145110 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:37:10.145110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:37:10.145119 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:37:10.146993 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:10.147508 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:10.147509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:10.147509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:10.147571 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:10.147857 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:10.147858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:10.147859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:10.148012 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:37:10.148087 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 14:37:10.148469 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 14:37:10.148566 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 14:37:10.149261 21824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:10.130977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:41353 (local address 127.2.103.67:40215)
0811 14:37:10.133959 (+ 2982us) negotiation.cc:107] Waiting for socket to connect
0811 14:37:10.134019 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 14:37:10.134824 (+ 805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:37:10.135230 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:37:10.135234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:37:10.135502 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:37:10.135887 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:10.135897 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.136939 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.136940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:10.138212 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:10.138215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.138758 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.138759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:10.138927 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:10.142181 (+ 3254us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:37:10.142193 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:37:10.144980 (+ 2787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:37:10.147096 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:37:10.147104 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:37:10.147106 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:37:10.147409 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:37:10.147646 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:37:10.147646 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:37:10.147647 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:37:10.147780 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:37:10.148101 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:37:10.148110 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:37:10.148289 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 14:37:10.148829 (+ 540us) client_negotiation.cc:240] Negotiation successful
0811 14:37:10.149009 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2922,"thread_start_us":93,"threads_started":1}
I20260811 14:37:10.149816 21821 heartbeater.cc:347] Connected to a master server at 127.2.103.126:41353
I20260811 14:37:10.149930 21821 heartbeater.cc:464] Registering TS with master...
I20260811 14:37:10.150147 21821 heartbeater.cc:511] Master 127.2.103.126:41353 requested a full tablet report, sending...
I20260811 14:37:10.150588 21055 ts_manager.cc:194] Registered new tserver with Master: 1ab6554aadad40c6a8a4a8f5e30e8814 (127.2.103.67:37959)
I20260811 14:37:10.151114 21055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:40215
I20260811 14:37:10.157958 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:37:10.168265 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:10.159789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32994 (local address 127.2.103.126:41353)
0811 14:37:10.159859 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 14:37:10.159861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:37:10.160119 (+ 258us) server_negotiation.cc:407] Connection header received
0811 14:37:10.160186 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:37:10.160187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:37:10.160199 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:37:10.160279 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:37:10.161023 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.161794 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.162927 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.163238 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.163306 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:10.164952 (+ 1646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:37:10.164954 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:37:10.164954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:37:10.164962 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:37:10.166855 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:10.167296 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:10.167297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:10.167298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:10.167364 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:10.167613 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:10.167614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:10.167615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:10.167777 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:37:10.167894 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 14:37:10.168060 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 14:37:10.168154 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 14:37:10.170378 21055 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32994:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:37:10.172916 21055 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:37:10.189244 21850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:10.175863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:45435 (local address 127.0.0.1:57668)
0811 14:37:10.182942 (+ 7079us) negotiation.cc:107] Waiting for socket to connect
0811 14:37:10.182955 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:37:10.183025 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:37:10.183487 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:37:10.183488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:37:10.183491 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:37:10.183541 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:10.183542 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.185234 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.185235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:10.187763 (+ 2528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:10.187767 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.188876 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.188877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:10.188984 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:10.189000 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 14:37:10.189072 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 14:37:10.189128 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7052,"mutex_wait_us":1333,"thread_start_us":92,"threads_started":1}
I20260811 14:37:10.189810 21853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:10.175998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57668 (local address 127.2.103.66:45435)
0811 14:37:10.182850 (+ 6852us) server_negotiation.cc:206] Beginning negotiation
0811 14:37:10.182856 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:37:10.183094 (+ 238us) server_negotiation.cc:407] Connection header received
0811 14:37:10.183144 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:37:10.183147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:37:10.183165 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:37:10.183261 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:37:10.183631 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.185125 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.187889 (+ 2764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.188678 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.188754 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:10.188836 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 14:37:10.189667 (+ 831us) server_negotiation.cc:299] Negotiation successful
0811 14:37:10.189718 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6790,"thread_start_us":95,"threads_started":1}
I20260811 14:37:10.190780 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:10.176353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33246 (local address 127.2.103.67:37959)
0811 14:37:10.183446 (+ 7093us) server_negotiation.cc:206] Beginning negotiation
0811 14:37:10.183451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:37:10.185686 (+ 2235us) server_negotiation.cc:407] Connection header received
0811 14:37:10.185748 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:37:10.185750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:37:10.185770 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:37:10.185847 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:37:10.186080 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.187650 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.189055 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.189993 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.190067 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:10.190195 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 14:37:10.190644 (+ 449us) server_negotiation.cc:299] Negotiation successful
0811 14:37:10.190691 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7045,"thread_start_us":99,"threads_started":1}
I20260811 14:37:10.190558 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:10.176117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:37959 (local address 127.0.0.1:33246)
0811 14:37:10.185538 (+ 9421us) negotiation.cc:107] Waiting for socket to connect
0811 14:37:10.185551 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:37:10.185613 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:37:10.185948 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:37:10.185949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:37:10.185951 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:37:10.185998 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:10.185999 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.187743 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.187744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:10.188904 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:10.188908 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.190234 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.190235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:10.190310 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:10.190316 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 14:37:10.190431 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 14:37:10.190476 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9397,"thread_start_us":76,"threads_started":1}
I20260811 14:37:10.191736 21717 tablet_service.cc:1467] Processing CreateTablet for tablet ddaf1ec65e1a474e8fea3a6e68a81137 (DEFAULT_TABLE table=test-table [id=828e67ff2d454e48b66880658654af5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:37:10.191809 21507 tablet_service.cc:1467] Processing CreateTablet for tablet ddaf1ec65e1a474e8fea3a6e68a81137 (DEFAULT_TABLE table=test-table [id=828e67ff2d454e48b66880658654af5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:37:10.192186 21507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddaf1ec65e1a474e8fea3a6e68a81137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:37:10.192188 21717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddaf1ec65e1a474e8fea3a6e68a81137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:37:10.195986 21860 tablet_bootstrap.cc:492] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a: Bootstrap starting.
I20260811 14:37:10.196779 21856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:10.176186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53830 (local address 127.2.103.65:35005)
0811 14:37:10.182736 (+ 6550us) server_negotiation.cc:206] Beginning negotiation
0811 14:37:10.182743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 14:37:10.182758 (+ 15us) server_negotiation.cc:407] Connection header received
0811 14:37:10.182808 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:37:10.182811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:37:10.182833 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:37:10.182941 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:37:10.183271 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.184475 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.186797 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.187569 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.187646 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:10.187704 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 14:37:10.196558 (+ 8854us) server_negotiation.cc:299] Negotiation successful
0811 14:37:10.196645 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6467,"thread_start_us":68,"threads_started":1}
I20260811 14:37:10.197093 21854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:10.175852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:35005 (local address 127.0.0.1:53830)
0811 14:37:10.176288 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 14:37:10.176331 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 14:37:10.176404 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:37:10.183059 (+ 6655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:37:10.183063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:37:10.183066 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:37:10.183166 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:10.183168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.184555 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.184556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:10.186708 (+ 2152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:10.186712 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.196290 (+ 9578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.196292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:10.196397 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:10.196406 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 14:37:10.196925 (+ 519us) client_negotiation.cc:240] Negotiation successful
0811 14:37:10.196994 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":43,"thread_start_us":107,"threads_started":1}
I20260811 14:37:10.197947 21308 tablet_service.cc:1467] Processing CreateTablet for tablet ddaf1ec65e1a474e8fea3a6e68a81137 (DEFAULT_TABLE table=test-table [id=828e67ff2d454e48b66880658654af5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:37:10.198378 21308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddaf1ec65e1a474e8fea3a6e68a81137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:37:10.198868 21860 tablet_bootstrap.cc:654] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a: Neither blocks nor log segments found. Creating new log.
I20260811 14:37:10.199513 21860 log.cc:824] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a: Log is configured to *not* fsync() on all Append() calls
I20260811 14:37:10.200409 21860 tablet_bootstrap.cc:492] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a: No bootstrap required, opened a new log
I20260811 14:37:10.200479 21860 ts_tablet_manager.cc:1397] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 14:37:10.202625 21860 raft_consensus.cc:348] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe511f326084fd8a38e61fedc1f6217" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35005 } } peers { permanent_uuid: "dad061a82a75441485384395e114037a" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45435 } } peers { permanent_uuid: "1ab6554aadad40c6a8a4a8f5e30e8814" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37959 } }
I20260811 14:37:10.202821 21860 raft_consensus.cc:374] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:37:10.202854 21860 raft_consensus.cc:729] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad061a82a75441485384395e114037a, State: Initialized, Role: FOLLOWER
I20260811 14:37:10.202970 21860 consensus_queue.cc:260] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe511f326084fd8a38e61fedc1f6217" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35005 } } peers { permanent_uuid: "dad061a82a75441485384395e114037a" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45435 } } peers { permanent_uuid: "1ab6554aadad40c6a8a4a8f5e30e8814" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37959 } }
I20260811 14:37:10.203243 21860 ts_tablet_manager.cc:1428] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:37:10.203666 21608 heartbeater.cc:503] Master 127.2.103.126:41353 was elected leader, sending a full tablet report...
I20260811 14:37:10.205327 21863 tablet_bootstrap.cc:492] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217: Bootstrap starting.
I20260811 14:37:10.206349 21863 tablet_bootstrap.cc:654] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217: Neither blocks nor log segments found. Creating new log.
I20260811 14:37:10.206782 21863 log.cc:824] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217: Log is configured to *not* fsync() on all Append() calls
I20260811 14:37:10.207652 21863 tablet_bootstrap.cc:492] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217: No bootstrap required, opened a new log
I20260811 14:37:10.207722 21863 ts_tablet_manager.cc:1397] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:37:10.210374 21863 raft_consensus.cc:348] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe511f326084fd8a38e61fedc1f6217" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35005 } } peers { permanent_uuid: "dad061a82a75441485384395e114037a" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45435 } } peers { permanent_uuid: "1ab6554aadad40c6a8a4a8f5e30e8814" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37959 } }
I20260811 14:37:10.210551 21863 raft_consensus.cc:374] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:37:10.210580 21863 raft_consensus.cc:729] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffe511f326084fd8a38e61fedc1f6217, State: Initialized, Role: FOLLOWER
I20260811 14:37:10.210815 21863 consensus_queue.cc:260] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe511f326084fd8a38e61fedc1f6217" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35005 } } peers { permanent_uuid: "dad061a82a75441485384395e114037a" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45435 } } peers { permanent_uuid: "1ab6554aadad40c6a8a4a8f5e30e8814" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37959 } }
I20260811 14:37:10.211228 21863 ts_tablet_manager.cc:1428] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:37:10.211393 21402 heartbeater.cc:503] Master 127.2.103.126:41353 was elected leader, sending a full tablet report...
I20260811 14:37:10.214854 21859 tablet_bootstrap.cc:492] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814: Bootstrap starting.
I20260811 14:37:10.216599 21859 tablet_bootstrap.cc:654] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814: Neither blocks nor log segments found. Creating new log.
I20260811 14:37:10.217013 21859 log.cc:824] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814: Log is configured to *not* fsync() on all Append() calls
I20260811 14:37:10.217880 21859 tablet_bootstrap.cc:492] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814: No bootstrap required, opened a new log
I20260811 14:37:10.217952 21859 ts_tablet_manager.cc:1397] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:37:10.220726 21859 raft_consensus.cc:348] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe511f326084fd8a38e61fedc1f6217" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35005 } } peers { permanent_uuid: "dad061a82a75441485384395e114037a" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45435 } } peers { permanent_uuid: "1ab6554aadad40c6a8a4a8f5e30e8814" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37959 } }
I20260811 14:37:10.220909 21859 raft_consensus.cc:374] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:37:10.220939 21859 raft_consensus.cc:729] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ab6554aadad40c6a8a4a8f5e30e8814, State: Initialized, Role: FOLLOWER
I20260811 14:37:10.221203 21859 consensus_queue.cc:260] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe511f326084fd8a38e61fedc1f6217" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35005 } } peers { permanent_uuid: "dad061a82a75441485384395e114037a" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45435 } } peers { permanent_uuid: "1ab6554aadad40c6a8a4a8f5e30e8814" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37959 } }
I20260811 14:37:10.221642 21859 ts_tablet_manager.cc:1428] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 14:37:10.221817 21821 heartbeater.cc:503] Master 127.2.103.126:41353 was elected leader, sending a full tablet report...
Aug 11 14:37:10 dist-test-slave-twwt krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459029, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
W20260811 14:37:10.309863 21403 tablet.cc:2369] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:37:10.335160 21610 tablet.cc:2369] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:37:10.381050 21822 tablet.cc:2369] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:37:10.431649 21867 raft_consensus.cc:482] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:37:10.431824 21867 raft_consensus.cc:504] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe511f326084fd8a38e61fedc1f6217" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35005 } } peers { permanent_uuid: "dad061a82a75441485384395e114037a" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45435 } } peers { permanent_uuid: "1ab6554aadad40c6a8a4a8f5e30e8814" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37959 } }
I20260811 14:37:10.432276 21867 leader_election.cc:288] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dad061a82a75441485384395e114037a (127.2.103.66:45435), 1ab6554aadad40c6a8a4a8f5e30e8814 (127.2.103.67:37959)
I20260811 14:37:10.436563 21853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:10.432953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:58741 (local address 127.2.103.66:45435)
0811 14:37:10.433008 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 14:37:10.433010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:37:10.433020 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:37:10.433053 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:37:10.433054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:37:10.433068 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:37:10.433139 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:37:10.433393 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.434140 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.435586 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.436132 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.436185 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:10.436196 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 14:37:10.436446 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 14:37:10.436491 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 14:37:10.436826 21873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:10.432476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:45435 (local address 127.2.103.65:58741)
0811 14:37:10.432755 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 14:37:10.432767 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:37:10.432829 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:37:10.433177 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:37:10.433178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:37:10.433181 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:37:10.433285 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:10.433287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.434247 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.434248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:10.435474 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:10.435478 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.436278 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.436279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:10.436355 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:10.436359 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:37:10.436695 (+ 336us) client_negotiation.cc:240] Negotiation successful
0811 14:37:10.436738 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":110,"threads_started":1}
I20260811 14:37:10.437176 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddaf1ec65e1a474e8fea3a6e68a81137" candidate_uuid: "ffe511f326084fd8a38e61fedc1f6217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad061a82a75441485384395e114037a" is_pre_election: true
I20260811 14:37:10.437373 21513 raft_consensus.cc:2454] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffe511f326084fd8a38e61fedc1f6217 in term 0.
I20260811 14:37:10.437673 21230 leader_election.cc:302] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dad061a82a75441485384395e114037a, ffe511f326084fd8a38e61fedc1f6217; no voters:
I20260811 14:37:10.438714 21867 raft_consensus.cc:2781] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:37:10.438784 21867 raft_consensus.cc:482] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:37:10.438810 21867 raft_consensus.cc:3037] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:37:10.439855 21867 raft_consensus.cc:504] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe511f326084fd8a38e61fedc1f6217" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35005 } } peers { permanent_uuid: "dad061a82a75441485384395e114037a" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45435 } } peers { permanent_uuid: "1ab6554aadad40c6a8a4a8f5e30e8814" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37959 } }
I20260811 14:37:10.440032 21867 leader_election.cc:288] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217 [CANDIDATE]: Term 1 election: Requested vote from peers dad061a82a75441485384395e114037a (127.2.103.66:45435), 1ab6554aadad40c6a8a4a8f5e30e8814 (127.2.103.67:37959)
I20260811 14:37:10.440253 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddaf1ec65e1a474e8fea3a6e68a81137" candidate_uuid: "ffe511f326084fd8a38e61fedc1f6217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad061a82a75441485384395e114037a"
I20260811 14:37:10.440333 21513 raft_consensus.cc:3037] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:37:10.441254 21513 raft_consensus.cc:2454] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffe511f326084fd8a38e61fedc1f6217 in term 1.
I20260811 14:37:10.441490 21230 leader_election.cc:302] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dad061a82a75441485384395e114037a, ffe511f326084fd8a38e61fedc1f6217; no voters:
I20260811 14:37:10.441610 21867 raft_consensus.cc:2781] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:37:10.441776 21867 raft_consensus.cc:686] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217 [term 1 LEADER]: Becoming Leader. State: Replica: ffe511f326084fd8a38e61fedc1f6217, State: Running, Role: LEADER
I20260811 14:37:10.441884 21867 consensus_queue.cc:237] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe511f326084fd8a38e61fedc1f6217" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35005 } } peers { permanent_uuid: "dad061a82a75441485384395e114037a" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45435 } } peers { permanent_uuid: "1ab6554aadad40c6a8a4a8f5e30e8814" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37959 } }
I20260811 14:37:10.442308 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:10.432844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:52615 (local address 127.2.103.67:37959)
0811 14:37:10.432896 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 14:37:10.432898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:37:10.433255 (+ 357us) server_negotiation.cc:407] Connection header received
0811 14:37:10.433356 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:37:10.433357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:37:10.433368 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:37:10.433398 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:37:10.433677 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.434694 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.440119 (+ 5425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.440718 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.440783 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:10.440792 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:37:10.442178 (+ 1386us) server_negotiation.cc:299] Negotiation successful
0811 14:37:10.442234 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 14:37:10.442945 21055 catalog_manager.cc:5495] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffe511f326084fd8a38e61fedc1f6217 (127.2.103.65). New cstate: current_term: 1 leader_uuid: "ffe511f326084fd8a38e61fedc1f6217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe511f326084fd8a38e61fedc1f6217" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dad061a82a75441485384395e114037a" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ab6554aadad40c6a8a4a8f5e30e8814" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37959 } health_report { overall_health: UNKNOWN } } }
I20260811 14:37:10.444301 21875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:10.432583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:37959 (local address 127.2.103.65:52615)
0811 14:37:10.433195 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0811 14:37:10.433206 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:37:10.433282 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:37:10.433464 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:37:10.433464 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:37:10.433466 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:37:10.433593 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:10.433595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.434879 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.434879 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:10.439960 (+ 5081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:10.439963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.441319 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.441320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:10.442037 (+ 717us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:10.442042 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:37:10.444149 (+ 2107us) client_negotiation.cc:240] Negotiation successful
0811 14:37:10.444199 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":95,"threads_started":1}
I20260811 14:37:10.444850 21753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddaf1ec65e1a474e8fea3a6e68a81137" candidate_uuid: "ffe511f326084fd8a38e61fedc1f6217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ab6554aadad40c6a8a4a8f5e30e8814" is_pre_election: true
I20260811 14:37:10.445010 21753 raft_consensus.cc:2454] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffe511f326084fd8a38e61fedc1f6217 in term 0.
I20260811 14:37:10.445138 21746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddaf1ec65e1a474e8fea3a6e68a81137" candidate_uuid: "ffe511f326084fd8a38e61fedc1f6217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ab6554aadad40c6a8a4a8f5e30e8814"
I20260811 14:37:10.445196 21746 raft_consensus.cc:3037] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:37:10.446178 21746 raft_consensus.cc:2454] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffe511f326084fd8a38e61fedc1f6217 in term 1.
I20260811 14:37:10.469516 21856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:10.466866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53840 (local address 127.2.103.65:35005)
0811 14:37:10.466930 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 14:37:10.466933 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:37:10.466970 (+ 37us) server_negotiation.cc:407] Connection header received
0811 14:37:10.467023 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:37:10.467024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:37:10.467050 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:37:10.467136 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:37:10.467318 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.467811 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.468609 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:10.468824 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:10.468917 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:10.469103 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:37:10.469247 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:37:10.469343 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 14:37:10.469362 (+ 19us) server_negotiation.cc:299] Negotiation successful
0811 14:37:10.469394 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 14:37:10.471511 21746 raft_consensus.cc:1261] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814 [term 1 FOLLOWER]: Refusing update from remote peer ffe511f326084fd8a38e61fedc1f6217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:37:10.471511 21513 raft_consensus.cc:1261] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a [term 1 FOLLOWER]: Refusing update from remote peer ffe511f326084fd8a38e61fedc1f6217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:37:10.471992 21878 consensus_queue.cc:1035] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ab6554aadad40c6a8a4a8f5e30e8814" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:37:10.472144 21867 consensus_queue.cc:1035] T ddaf1ec65e1a474e8fea3a6e68a81137 P ffe511f326084fd8a38e61fedc1f6217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad061a82a75441485384395e114037a" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:37:10.477443 21886 mvcc.cc:205] Tried to move back new op lower bound from 7317336188808269824 to 7317336188690386944. Current Snapshot: MvccSnapshot[applied={T|T < 7317336188808269824}]
I20260811 14:37:10.476958 21888 mvcc.cc:205] Tried to move back new op lower bound from 7317336188808269824 to 7317336188690386944. Current Snapshot: MvccSnapshot[applied={T|T < 7317336188808269824 or (T in {7317336188808269824})}]
I20260811 14:37:10.478009 21885 mvcc.cc:205] Tried to move back new op lower bound from 7317336188808269824 to 7317336188690386944. Current Snapshot: MvccSnapshot[applied={T|T < 7317336188808269824}]
Aug 11 14:37:10 dist-test-slave-twwt krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459029, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
I20260811 14:37:10.492151 21055 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:37:10.492292 21055 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:37:10.493330 21055 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3cf4bca4fd54c8dbd44d69dee771ef5: Sending DeleteTablet for 3 replicas of tablet ddaf1ec65e1a474e8fea3a6e68a81137
I20260811 14:37:10.493759 21717 tablet_service.cc:1514] Processing DeleteTablet for tablet ddaf1ec65e1a474e8fea3a6e68a81137 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:37:10 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:33246
I20260811 14:37:10.494251 21507 tablet_service.cc:1514] Processing DeleteTablet for tablet ddaf1ec65e1a474e8fea3a6e68a81137 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:37:10 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:57668
I20260811 14:37:10.494479 21308 tablet_service.cc:1514] Processing DeleteTablet for tablet ddaf1ec65e1a474e8fea3a6e68a81137 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:37:10 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:53830
I20260811 14:37:10.495739 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 21196
I20260811 14:37:10.508327 21897 tablet_replica.cc:330] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814: stopping tablet replica
I20260811 14:37:10.508517 21897 raft_consensus.cc:2229] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:37:10.508677 21897 raft_consensus.cc:2258] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:37:10.509052 21898 tablet_replica.cc:330] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a: stopping tablet replica
I20260811 14:37:10.509194 21898 raft_consensus.cc:2229] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:37:10.509323 21898 raft_consensus.cc:2258] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:37:10.509500 21897 ts_tablet_manager.cc:1905] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:37:10.510963 21898 ts_tablet_manager.cc:1905] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:37:10.512596 21898 ts_tablet_manager.cc:1918] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:37:10.512776 21898 log.cc:1196] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/ddaf1ec65e1a474e8fea3a6e68a81137
I20260811 14:37:10.513127 21898 ts_tablet_manager.cc:1939] T ddaf1ec65e1a474e8fea3a6e68a81137 P dad061a82a75441485384395e114037a: Deleting consensus metadata
I20260811 14:37:10.513684 21035 catalog_manager.cc:4841] TS dad061a82a75441485384395e114037a (127.2.103.66:45435): tablet ddaf1ec65e1a474e8fea3a6e68a81137 (table test-table [id=828e67ff2d454e48b66880658654af5a]) successfully deleted
I20260811 14:37:10.515970 21897 ts_tablet_manager.cc:1918] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260811 14:37:10.515986 21034 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.103.65:35005 (error 108)
I20260811 14:37:10.516064 21897 log.cc:1196] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/ddaf1ec65e1a474e8fea3a6e68a81137
I20260811 14:37:10.516690 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 21408
I20260811 14:37:10.531879 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:10.516851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:35005 (local address 127.0.0.1:53844)
0811 14:37:10.531731 (+ 14880us) negotiation.cc:107] Waiting for socket to connect
0811 14:37:10.531816 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.65:35005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14839}
W20260811 14:37:10.533366 21034 catalog_manager.cc:4584] TS ffe511f326084fd8a38e61fedc1f6217 (127.2.103.65:35005): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ddaf1ec65e1a474e8fea3a6e68a81137: Network error: Client connection negotiation failed: client connection to 127.2.103.65:35005: connect: Connection refused (error 111)
I20260811 14:37:10.538394 21897 ts_tablet_manager.cc:1939] T ddaf1ec65e1a474e8fea3a6e68a81137 P 1ab6554aadad40c6a8a4a8f5e30e8814: Deleting consensus metadata
I20260811 14:37:10.539135 21036 catalog_manager.cc:4841] TS 1ab6554aadad40c6a8a4a8f5e30e8814 (127.2.103.67:37959): tablet ddaf1ec65e1a474e8fea3a6e68a81137 (table test-table [id=828e67ff2d454e48b66880658654af5a]) successfully deleted
I20260811 14:37:10.556217 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 21617
I20260811 14:37:10.585168 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 21005
I20260811 14:37:10.602850 2461 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 14:37:10.957828 2461 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 14:37:10.957943 2461 mini_ranger.cc:66] Stopping Ranger...
I20260811 14:37:11.313207 2461 mini_ranger.cc:68] Stopped Ranger
2026-08-11 14:37:11.313 UTC [20358] LOG: received smart shutdown request
2026-08-11 14:37:11.318 UTC [20358] LOG: background worker "logical replication launcher" (PID 20373) exited with exit code 1
2026-08-11 14:37:11.318 UTC [20365] LOG: shutting down
2026-08-11 14:37:11.496 UTC [20358] LOG: database system is shut down
2026-08-11T14:37:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50407 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:37:11 dist-test-slave-twwt krb5kdc[21917](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:37:11 dist-test-slave-twwt krb5kdc[21917](info): set up 2 sockets
Aug 11 14:37:11 dist-test-slave-twwt krb5kdc[21917](info): commencing operation
krb5kdc: starting...
W20260811 14:37:12.896080 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:37:12 dist-test-slave-twwt krb5kdc[21917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:37:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:37:12Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.2.103.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.2.103.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.2.103.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/rangeradmin_127.2.103.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.2.103.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/rangeradmin_127.2.103.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.2.103.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.2.103.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.2.103.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/rangerlookup_127.2.103.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.2.103.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/rangerlookup_127.2.103.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.103.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.103.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.103.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/HTTP_127.2.103.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.103.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/HTTP_127.2.103.84@KRBTEST.COM.keytab.
I20260811 14:37:13.079474 2461 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskmCSHXK/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/postgres -l logfile start
2026-08-11 14:37:15.671 UTC [21961] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 14:37:15.671 UTC [21961] LOG: listening on IPv4 address "127.2.103.84", port 44533
2026-08-11 14:37:15.675 UTC [21961] LOG: listening on Unix socket "/tmp/.s.PGSQL.44533"
2026-08-11 14:37:15.697 UTC [21964] LOG: database system was shut down at 2026-08-11 14:37:14 UTC
2026-08-11 14:37:15.701 UTC [21961] LOG: database system is ready to accept connections
I20260811 14:37:16.903652 2461 mini_postgres.cc:96] Postgres bound to 44533
2026-08-11 14:37:16.911 UTC [21976] FATAL: database "slave" does not exist
127.2.103.84:44533 - accepting connections
I20260811 14:37:16.912002 2461 mini_ranger.cc:161] Starting Ranger...
I20260811 14:37:16.928828 2461 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 14:37:17.454802 2461 mini_ranger.cc:87] Created ranger Postgres database
I20260811 14:37:17.454926 2461 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ranger-admin
2026-08-11 14:37:17,753 [I] DB FLAVOR :POSTGRES
2026-08-11 14:37:17,753 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 14:37:17,753 [I] Checking connection..
2026-08-11 14:37:17,753 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 14:37:18,027 [I] Checking connection passed.
2026-08-11 14:37:18,027 [I] --------- Verifying version history table ---------
2026-08-11 14:37:18,028 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 14:37:18,292 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 14:37:18,293 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 14:37:18,574 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 14:37:18,574 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 14:37:18,574 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 14:37:18.828 UTC [22070] WARNING: there is no transaction in progress
2026-08-11 14:37:18,837 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 14:37:18,838 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 14:37:19,106 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 14:37:19,107 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 14:37:19,107 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 14:37:19,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 14:37:19,617 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-twwt.c.gcp-upstream.internal','N') ;"
2026-08-11 14:37:19,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 14:37:20,128 [I] Table x_portal_user does not exist in database ranger
2026-08-11 14:37:20,128 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 14:37:20,389 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 14:37:20,390 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 14:37:20,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 14:37:20,945 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 14:37:20,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 14:37:21.997 UTC [22229] WARNING: there is no transaction in progress
2026-08-11 14:37:22.015 UTC [22229] WARNING: there is no transaction in progress
2026-08-11 14:37:22.034 UTC [22229] WARNING: there is no transaction in progress
2026-08-11 14:37:22.051 UTC [22229] WARNING: there is no transaction in progress
2026-08-11 14:37:22.067 UTC [22229] WARNING: there is no transaction in progress
2026-08-11 14:37:22.085 UTC [22229] WARNING: there is no transaction in progress
2026-08-11 14:37:22.143 UTC [22229] WARNING: there is no transaction in progress
2026-08-11 14:37:22.150 UTC [22229] WARNING: there is no transaction in progress
2026-08-11 14:37:22.167 UTC [22229] WARNING: there is no transaction in progress
2026-08-11 14:37:22.179 UTC [22229] WARNING: there is no transaction in progress
2026-08-11 14:37:22.190 UTC [22229] WARNING: there is no transaction in progress
2026-08-11 14:37:22.205 UTC [22229] WARNING: there is no transaction in progress
2026-08-11 14:37:22.215 UTC [22229] WARNING: there is no transaction in progress
2026-08-11 14:37:22.222 UTC [22229] WARNING: there is no transaction in progress
2026-08-11 14:37:22.229 UTC [22229] WARNING: there is no transaction in progress
2026-08-11 14:37:22.236 UTC [22229] WARNING: there is no transaction in progress
2026-08-11 14:37:22,952 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 14:37:22,952 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 14:37:23,192 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 14:37:23,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 14:37:23,450 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 14:37:23,451 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 14:37:23,730 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 14:37:23,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 14:37:23,996 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 14:37:24,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-twwt.c.gcp-upstream.internal';"
2026-08-11 14:37:24,541 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 14:37:24,542 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 14:37:24,815 [I] DB_PATCHES have already been applied
I20260811 14:37:24.822198 2461 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ranger-admin:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskmCSHXK/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 14:37:24.822270 2461 mini_ranger.cc:193] Using host: 127.2.103.84
I20260811 14:37:24.826494 2461 mini_ranger.cc:227] Ranger admin URL: http://127.2.103.84:50409
Aug 11, 2026 2:37:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 2:37:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 2:37:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 2:37:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 2:37:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 2:37:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 2:37:25 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 14:37:26.258492 2461 mini_ranger.cc:160] Time spent starting Ranger: real 9.346s user 0.003s sys 0.009s
Aug 11, 2026 2:37:46 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 2:37:46 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 2:37:46 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 2:37:46 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 2:37:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 2:37:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 2:37:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 2:37:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 2:37:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 2:37:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 2:37:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 2:37:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 2:37:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 2:37:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 2:37:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 2:37:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 2:37:46 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 2:37:47 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 14:37:48.277341 2461 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.2.103.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.2.103.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.2.103.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/rangerkms_127.2.103.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.2.103.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/rangerkms_127.2.103.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.103.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.103.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.103.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/HTTP_127.2.103.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.103.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/HTTP_127.2.103.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 14:37:48 dist-test-slave-twwt krb5kdc[21917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459068, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 14:37:48.371361 2461 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 14:37:48.382840 2461 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 14:37:49.344049 2461 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 14:37:49.344149 2461 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ranger-kms
I20260811 14:37:49.344169 2461 mini_ranger_kms.cc:227] Using postgres at 127.2.103.84:44533
2026-08-11 14:37:49,532 [I] DB FLAVOR :POSTGRES
2026-08-11 14:37:49,533 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 14:37:49,533 [I] Checking connection
2026-08-11 14:37:49,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 14:37:49,787 [I] connection success
2026-08-11 14:37:49,787 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 14:37:49,787 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 14:37:49,787 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 14:37:50,056 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 14:37:50,057 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 14:37:50,057 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 14:37:50,057 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.103.84:44533/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 14:37:50,330 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 14:37:50.811772 2461 mini_ranger_kms.cc:310] Created kms service
I20260811 14:37:51.031438 2461 mini_ranger_kms.cc:326] Created kudu user
I20260811 14:37:51.078482 2461 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 14:37:51 dist-test-slave-twwt krb5kdc[21917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459068, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
I20260811 14:37:51.513422 2461 mini_ranger_kms.cc:384] Added ranger policy
I20260811 14:37:51.513541 2461 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ranger-kms:/tmp/dist-test-taskmCSHXK/build/release/bin/postgresql.jar:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskmCSHXK/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 14:37:51.513561 2461 mini_ranger_kms.cc:240] Using host: 127.2.103.84
I20260811 14:37:51.518419 2461 mini_ranger_kms.cc:276] Ranger KMS PID: 22492
I20260811 14:37:51.518500 2461 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.2.103.84:43053
Aug 11, 2026 2:37:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 2:37:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 2:37:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 2:37:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 2:37:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 2:37:53 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 14:37:53 dist-test-slave-twwt krb5kdc[21917](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786459073, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.103.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 14:37:54 dist-test-slave-twwt krb5kdc[21917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459073, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.103.84@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
I20260811 14:37:54.525456 2461 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.154s user 0.008s sys 0.161s
I20260811 14:37:54.525555 2461 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 14:37:54.525599 2461 mini_ranger_kms.cc:401] 127.2.103.84:43053/kms/v1/keys
Aug 11 14:37:54 dist-test-slave-twwt krb5kdc[21917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459073, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.103.84@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
Aug 11 14:37:57 dist-test-slave-twwt krb5kdc[21917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:37:57.213310 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:36973
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:41313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:36973
--ranger_config_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.103.84:43053/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:37:57.300630 22571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:37:57.300853 22571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:37:57.300875 22571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:37:57.302057 22571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:37:57.302102 22571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:37:57.302116 22571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:37:57.302129 22571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:37:57.302181 22571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:37:57.303802 22571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:41313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/wal
--ranger_kms_url=127.2.103.84:43053/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:36973
--ranger_config_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:36973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:37:57.304081 22571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:37:57.304320 22571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:37:57.309557 22583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:37:57.309567 22578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:37:57.309553 22580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:37:57.309893 22571 server_base.cc:1034] running on GCE node
I20260811 14:37:57.310144 22571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:37:57.310441 22571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:37:57.311586 22571 hybrid_clock.cc:648] HybridClock initialized: now 1786459077311564 us; error 36 us; skew 500 ppm
Aug 11 14:37:57 dist-test-slave-twwt krb5kdc[21917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459077, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:37:57.319437 22571 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:37:57.320004 22571 webserver.cc:460] Webserver started at http://127.2.103.126:42749/ using document root <none> and password file <none>
I20260811 14:37:57.320283 22571 fs_manager.cc:359] Metadata directory not provided
I20260811 14:37:57.320330 22571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:37:57.320497 22571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 14:37:57 dist-test-slave-twwt krb5kdc[21917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459077, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
I20260811 14:37:57.369132 22571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "8297799e63e7413db1865893811513f2"
format_stamp: "Formatted at 2026-08-11 14:37:57 on dist-test-slave-twwt"
server_key: "e7140a63fc69bc6ec61f76b9455c9101"
server_key_iv: "ce131e0edbe47ac7c734d15e5250d42d"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:57.369602 22571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "8297799e63e7413db1865893811513f2"
format_stamp: "Formatted at 2026-08-11 14:37:57 on dist-test-slave-twwt"
server_key: "e7140a63fc69bc6ec61f76b9455c9101"
server_key_iv: "ce131e0edbe47ac7c734d15e5250d42d"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:57.371492 22571 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 14:37:57.389117 22597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:57.389427 22571 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:37:57.389539 22571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "8297799e63e7413db1865893811513f2"
format_stamp: "Formatted at 2026-08-11 14:37:57 on dist-test-slave-twwt"
server_key: "e7140a63fc69bc6ec61f76b9455c9101"
server_key_iv: "ce131e0edbe47ac7c734d15e5250d42d"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:57.389648 22571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:37:57.399268 22571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:37:57.401271 22571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:37:57.401453 22571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:37:57.406641 22571 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:36973
I20260811 14:37:57.406677 22666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:36973 every 8 connection(s)
I20260811 14:37:57.407058 22571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:37:57.407963 22667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:37:57.410970 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 22571
I20260811 14:37:57.411093 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:37:57.411285 2461 external_mini_cluster.cc:1428] Setting key cd3e2049d6439644ec355c936f76bb2b
I20260811 14:37:57.411289 22667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2: Bootstrap starting.
I20260811 14:37:57.412038 22667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2: Neither blocks nor log segments found. Creating new log.
I20260811 14:37:57.412493 22667 log.cc:824] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2: Log is configured to *not* fsync() on all Append() calls
I20260811 14:37:57.413450 22667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2: No bootstrap required, opened a new log
I20260811 14:37:57.416074 22667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8297799e63e7413db1865893811513f2" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36973 } }
I20260811 14:37:57.416263 22667 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:37:57.416297 22667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8297799e63e7413db1865893811513f2, State: Initialized, Role: FOLLOWER
I20260811 14:37:57.416486 22667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8297799e63e7413db1865893811513f2" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36973 } }
I20260811 14:37:57.416572 22667 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:37:57.416602 22667 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:37:57.416669 22667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:37:57.417464 22667 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8297799e63e7413db1865893811513f2" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36973 } }
I20260811 14:37:57.417591 22667 leader_election.cc:302] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8297799e63e7413db1865893811513f2; no voters:
I20260811 14:37:57.417822 22667 leader_election.cc:288] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:37:57.417882 22672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:37:57.418154 22672 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [term 1 LEADER]: Becoming Leader. State: Replica: 8297799e63e7413db1865893811513f2, State: Running, Role: LEADER
I20260811 14:37:57.418244 22667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:37:57.418347 22672 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8297799e63e7413db1865893811513f2" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36973 } }
Aug 11 14:37:57 dist-test-slave-twwt krb5kdc[21917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:37:57.418833 22672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8297799e63e7413db1865893811513f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8297799e63e7413db1865893811513f2" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36973 } } }
I20260811 14:37:57.418906 22672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [sys.catalog]: This master's current role is: LEADER
I20260811 14:37:57.419062 22672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8297799e63e7413db1865893811513f2. Latest consensus state: current_term: 1 leader_uuid: "8297799e63e7413db1865893811513f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8297799e63e7413db1865893811513f2" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36973 } } }
I20260811 14:37:57.419171 22672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [sys.catalog]: This master's current role is: LEADER
I20260811 14:37:57.419368 22683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:37:57.420395 22683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:37:57.420751 22667 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 14:37:57.422156 22694 mvcc.cc:205] Tried to move back new op lower bound from 7317336381118582784 to 7317336381106241536. Current Snapshot: MvccSnapshot[applied={T|T < 7317336381118582784}]
I20260811 14:37:57.422705 22683 catalog_manager.cc:1347] Generated new cluster ID: 118a9a9241c44cda986b352d758c6e11
I20260811 14:37:57.422760 22683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:37:57.424122 22670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:57.411943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51010 (local address 127.2.103.126:36973)
0811 14:37:57.412142 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 14:37:57.412151 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:37:57.412183 (+ 32us) server_negotiation.cc:407] Connection header received
0811 14:37:57.412775 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:37:57.412786 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:37:57.412872 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:37:57.413044 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:37:57.414079 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:57.415077 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:57.415852 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:57.416314 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:57.416419 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:57.419570 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:37:57.419575 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:37:57.419575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:37:57.419590 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:37:57.421854 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:57.422364 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:57.422367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:57.422368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:57.422471 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:57.422788 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:57.422789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:57.422789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:57.423106 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:37:57.423249 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 14:37:57.423779 (+ 530us) server_negotiation.cc:299] Negotiation successful
0811 14:37:57.423914 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":71,"threads_started":1}
I20260811 14:37:57.429378 22683 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:37:57.429960 22683 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:37:57.436926 22683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2: Generated new TSK 0
I20260811 14:37:57.437391 22683 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
I20260811 14:37:58.153959 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.103.84:43053/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:36973
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:41313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:37:58.281239 22728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:37:58.281543 22728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:37:58.281577 22728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:37:58.283068 22728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:37:58.283123 22728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:37:58.283154 22728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:37:58.284900 22728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:41313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.2.103.84:43053/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:36973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:37:58.285171 22728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:37:58.285451 22728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:37:58.291865 22737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:37:58.291872 22739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:37:58.292016 22735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:37:58.292223 22728 server_base.cc:1034] running on GCE node
I20260811 14:37:58.292606 22728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:37:58.293032 22728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:37:58.294229 22728 hybrid_clock.cc:648] HybridClock initialized: now 1786459078294202 us; error 34 us; skew 500 ppm
Aug 11 14:37:58 dist-test-slave-twwt krb5kdc[21917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459078, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:37:58.297310 22728 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:37:58.297844 22728 webserver.cc:460] Webserver started at http://127.2.103.65:42091/ using document root <none> and password file <none>
I20260811 14:37:58.298110 22728 fs_manager.cc:359] Metadata directory not provided
I20260811 14:37:58.298182 22728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:37:58.298348 22728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 14:37:58 dist-test-slave-twwt krb5kdc[21917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459078, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
I20260811 14:37:58.314970 22728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "1c7a8e78377d4355a9568dd688d9aa91"
format_stamp: "Formatted at 2026-08-11 14:37:58 on dist-test-slave-twwt"
server_key: "e61c9faf165c97b7be0cfda68ee88c7a"
server_key_iv: "856dfe821aa0434818ab9490d817474d"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:58.315568 22728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "1c7a8e78377d4355a9568dd688d9aa91"
format_stamp: "Formatted at 2026-08-11 14:37:58 on dist-test-slave-twwt"
server_key: "e61c9faf165c97b7be0cfda68ee88c7a"
server_key_iv: "856dfe821aa0434818ab9490d817474d"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:58.317691 22728 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260811 14:37:58.331893 22750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:58.332180 22728 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:37:58.332309 22728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "1c7a8e78377d4355a9568dd688d9aa91"
format_stamp: "Formatted at 2026-08-11 14:37:58 on dist-test-slave-twwt"
server_key: "e61c9faf165c97b7be0cfda68ee88c7a"
server_key_iv: "856dfe821aa0434818ab9490d817474d"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:58.332432 22728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:37:58.368722 22728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:37:58.371058 22728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:37:58.371258 22728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:37:58.371528 22728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:37:58.371927 22728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:37:58.371981 22728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:58.372051 22728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:37:58.372097 22728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:58.379907 22728 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:37127
I20260811 14:37:58.379987 22936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:37127 every 8 connection(s)
I20260811 14:37:58.380388 22728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:37:58.383198 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 22728
I20260811 14:37:58.383337 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:37:58.383529 2461 external_mini_cluster.cc:1428] Setting key cc36b5853c76bd9d9426d78ca4c2a650
Aug 11 14:37:58 dist-test-slave-twwt krb5kdc[21917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459078, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:37:58.392670 22940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:58.381352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:45717 (local address 127.2.103.126:36973)
0811 14:37:58.381517 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 14:37:58.381520 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:37:58.382894 (+ 1374us) server_negotiation.cc:407] Connection header received
0811 14:37:58.383814 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:37:58.383815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:37:58.383832 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:37:58.383912 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:37:58.384687 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:58.385480 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:58.386902 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:58.387127 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:58.387228 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:58.389444 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:37:58.389445 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:37:58.389446 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:37:58.389454 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:37:58.391100 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:58.391561 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:58.391562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:58.391563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:58.391629 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:58.391904 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:58.391905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:58.391906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:58.392031 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:37:58.392105 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 14:37:58.392461 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 14:37:58.392561 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":109,"threads_started":1}
I20260811 14:37:58.392864 22941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:58.381340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:36973 (local address 127.2.103.65:45717)
0811 14:37:58.382752 (+ 1412us) negotiation.cc:107] Waiting for socket to connect
0811 14:37:58.382794 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 14:37:58.383583 (+ 789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:37:58.383999 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:37:58.384003 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:37:58.384260 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:37:58.384567 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:58.384575 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:58.385605 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:58.385606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:58.386814 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:58.386818 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:58.387254 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:58.387255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:58.387380 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:58.387953 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:37:58.387963 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:37:58.389340 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:37:58.391227 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:37:58.391235 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:37:58.391237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:37:58.391480 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:37:58.391702 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:37:58.391703 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:37:58.391703 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:37:58.391827 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:37:58.392105 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:37:58.392114 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:37:58.392298 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 14:37:58.392421 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 14:37:58.392584 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1362,"thread_start_us":109,"threads_started":1}
I20260811 14:37:58.393738 22938 heartbeater.cc:347] Connected to a master server at 127.2.103.126:36973
I20260811 14:37:58.393872 22938 heartbeater.cc:464] Registering TS with master...
I20260811 14:37:58.394088 22938 heartbeater.cc:511] Master 127.2.103.126:36973 requested a full tablet report, sending...
I20260811 14:37:58.394748 22621 ts_manager.cc:194] Registered new tserver with Master: 1c7a8e78377d4355a9568dd688d9aa91 (127.2.103.65:37127)
I20260811 14:37:58.395392 22621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:45717
WARNING: no policy specified for kudu/127.2.103.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
I20260811 14:37:58.421486 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.103.84:43053/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:36973
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:41313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 14:37:58 dist-test-slave-twwt krb5kdc[21917](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786459078, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 14:37:58.545758 22949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:37:58.546133 22949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:37:58.546227 22949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:37:58.547969 22949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:37:58.548035 22949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:37:58.548074 22949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:37:58.550244 22949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:41313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.2.103.84:43053/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:36973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:37:58.550747 22949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:37:58.551075 22949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:37:58.557883 22963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:37:58.558184 22971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:37:58.559090 22964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:37:58.561645 22949 server_base.cc:1034] running on GCE node
I20260811 14:37:58.561932 22949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:37:58.562237 22949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:37:58.563418 22949 hybrid_clock.cc:648] HybridClock initialized: now 1786459078563389 us; error 36 us; skew 500 ppm
Aug 11 14:37:58 dist-test-slave-twwt krb5kdc[21917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459078, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:37:58.567353 22949 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:37:58.567873 22949 webserver.cc:460] Webserver started at http://127.2.103.66:35965/ using document root <none> and password file <none>
I20260811 14:37:58.568138 22949 fs_manager.cc:359] Metadata directory not provided
I20260811 14:37:58.568185 22949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:37:58.568308 22949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 14:37:58 dist-test-slave-twwt krb5kdc[21917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459078, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
I20260811 14:37:58.593276 22949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "619ce6b5451f48149031f2f7223a0fec"
format_stamp: "Formatted at 2026-08-11 14:37:58 on dist-test-slave-twwt"
server_key: "7ef193971188a266f8a0faecc6fd0037"
server_key_iv: "0d7dbfe75ab761c4d436cd4588d6d26c"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:58.593917 22949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "619ce6b5451f48149031f2f7223a0fec"
format_stamp: "Formatted at 2026-08-11 14:37:58 on dist-test-slave-twwt"
server_key: "7ef193971188a266f8a0faecc6fd0037"
server_key_iv: "0d7dbfe75ab761c4d436cd4588d6d26c"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:58.596082 22949 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 14:37:58.609736 22979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:58.610088 22949 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 14:37:58.610595 22949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "619ce6b5451f48149031f2f7223a0fec"
format_stamp: "Formatted at 2026-08-11 14:37:58 on dist-test-slave-twwt"
server_key: "7ef193971188a266f8a0faecc6fd0037"
server_key_iv: "0d7dbfe75ab761c4d436cd4588d6d26c"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:58.610940 22949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:37:58.654752 22949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:37:58.657413 22949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:37:58.657647 22949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:37:58.658051 22949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:37:58.658514 22949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:37:58.658639 22949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:58.658718 22949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:37:58.658973 22949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:58.666150 22949 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:34585
I20260811 14:37:58.666867 22949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:37:58.670432 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 22949
I20260811 14:37:58.670547 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:37:58.670796 2461 external_mini_cluster.cc:1428] Setting key 54dbb9bd3ba2884cd28ad0c6ecd72a1d
I20260811 14:37:58.673198 23160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:34585 every 8 connection(s)
WARNING: no policy specified for kudu/127.2.103.67@KRBTEST.COM; defaulting to no policy
Aug 11 14:37:58 dist-test-slave-twwt krb5kdc[21917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459078, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.67@KRBTEST.COM" created.
I20260811 14:37:58.704020 22940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:58.679649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:41523 (local address 127.2.103.126:36973)
0811 14:37:58.679697 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 14:37:58.679699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:37:58.686950 (+ 7251us) server_negotiation.cc:407] Connection header received
0811 14:37:58.687810 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:37:58.687812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:37:58.687833 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:37:58.687868 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:37:58.688800 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:58.689598 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:58.691164 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:58.691489 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:58.691538 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:58.700469 (+ 8931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:37:58.700471 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:37:58.700471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:37:58.700480 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:37:58.702286 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:58.702877 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:58.702878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:58.702879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:58.702940 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:58.703228 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:58.703229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:58.703229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:58.703388 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:37:58.703441 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 14:37:58.703816 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 14:37:58.703916 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:37:58.704870 23165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:58.679307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:36973 (local address 127.2.103.66:41523)
0811 14:37:58.686748 (+ 7441us) negotiation.cc:107] Waiting for socket to connect
0811 14:37:58.686821 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 14:37:58.687654 (+ 833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:37:58.688094 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:37:58.688099 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:37:58.688371 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:37:58.688676 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:58.688684 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:58.689726 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:58.689727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:58.691057 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:58.691061 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:58.691650 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:58.691652 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:58.691808 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:58.692444 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:37:58.692455 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:37:58.700306 (+ 7851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:37:58.702410 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:37:58.702420 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:37:58.702423 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:37:58.702784 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:37:58.703007 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:37:58.703008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:37:58.703008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:37:58.703155 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:37:58.703477 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:37:58.703484 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:37:58.703663 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 14:37:58.704424 (+ 761us) client_negotiation.cc:240] Negotiation successful
0811 14:37:58.704614 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7372,"thread_start_us":117,"threads_started":1}
I20260811 14:37:58.705617 23161 heartbeater.cc:347] Connected to a master server at 127.2.103.126:36973
I20260811 14:37:58.705720 23161 heartbeater.cc:464] Registering TS with master...
I20260811 14:37:58.705930 23161 heartbeater.cc:511] Master 127.2.103.126:36973 requested a full tablet report, sending...
I20260811 14:37:58.706387 22621 ts_manager.cc:194] Registered new tserver with Master: 619ce6b5451f48149031f2f7223a0fec (127.2.103.66:34585)
I20260811 14:37:58.706982 22621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:41523
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
I20260811 14:37:58.726889 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.103.84:43053/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:36973
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:41313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:37:58.842178 23170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:37:58.842420 23170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:37:58.842445 23170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:37:58.843752 23170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:37:58.843804 23170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:37:58.843833 23170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:37:58.845424 23170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:41313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.2.103.84:43053/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:36973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:37:58.845705 23170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:37:58.845988 23170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:37:58.855006 23181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:37:58.857961 23179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:37:58.858246 23170 server_base.cc:1034] running on GCE node
W20260811 14:37:58.858523 23183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:37:58.858800 23170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:37:58.859126 23170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:37:58.860339 23170 hybrid_clock.cc:648] HybridClock initialized: now 1786459078860312 us; error 43 us; skew 500 ppm
Aug 11 14:37:58 dist-test-slave-twwt krb5kdc[21917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459078, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:37:58.863583 23170 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:37:58.864099 23170 webserver.cc:460] Webserver started at http://127.2.103.67:40559/ using document root <none> and password file <none>
I20260811 14:37:58.864373 23170 fs_manager.cc:359] Metadata directory not provided
I20260811 14:37:58.864445 23170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:37:58.864599 23170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 14:37:58 dist-test-slave-twwt krb5kdc[21917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459078, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
I20260811 14:37:58.877452 23170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "c64d600ac05f484b88d41fe8dab2facb"
format_stamp: "Formatted at 2026-08-11 14:37:58 on dist-test-slave-twwt"
server_key: "51ca9c0227ddf729647484940f9ccb0a"
server_key_iv: "56790d8d5df2b52363aec79856c0d224"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:58.877976 23170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "c64d600ac05f484b88d41fe8dab2facb"
format_stamp: "Formatted at 2026-08-11 14:37:58 on dist-test-slave-twwt"
server_key: "51ca9c0227ddf729647484940f9ccb0a"
server_key_iv: "56790d8d5df2b52363aec79856c0d224"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:58.879843 23170 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260811 14:37:58.901319 23192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:58.901525 23170 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 14:37:58.901631 23170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "c64d600ac05f484b88d41fe8dab2facb"
format_stamp: "Formatted at 2026-08-11 14:37:58 on dist-test-slave-twwt"
server_key: "51ca9c0227ddf729647484940f9ccb0a"
server_key_iv: "56790d8d5df2b52363aec79856c0d224"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:58.901747 23170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:37:58.916541 23170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:37:58.919456 23170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:37:58.919688 23170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:37:58.919934 23170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:37:58.922636 23170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:37:58.922741 23170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:58.922820 23170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:37:58.922894 23170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:58.930711 23170 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:36309
I20260811 14:37:58.931133 23170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:37:58.931703 23359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:36309 every 8 connection(s)
I20260811 14:37:58.937199 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 23170
I20260811 14:37:58.937340 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:37:58.937538 2461 external_mini_cluster.cc:1428] Setting key 7be0b6280df7dd034e5eaebe25b6e120
Aug 11 14:37:58 dist-test-slave-twwt krb5kdc[21917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459078, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:37:58.941145 22940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:58.932382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:36331 (local address 127.2.103.126:36973)
0811 14:37:58.932429 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 14:37:58.932431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:37:58.932442 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:37:58.932925 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:37:58.932926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:37:58.932940 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:37:58.932999 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:37:58.933696 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:58.934216 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:58.935311 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:58.935526 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:58.935606 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:58.938028 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:37:58.938029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:37:58.938029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:37:58.938038 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:37:58.939718 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:58.940137 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:58.940138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:58.940139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:58.940203 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:58.940471 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:58.940472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:58.940472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:58.940595 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:37:58.940664 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 14:37:58.940969 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 14:37:58.941054 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:37:58.941537 23362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:58.931986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:36973 (local address 127.2.103.67:36331)
0811 14:37:58.932180 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 14:37:58.932225 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 14:37:58.932800 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:37:58.933083 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:37:58.933087 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:37:58.933288 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:37:58.933597 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:58.933603 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:58.934323 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:58.934325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:58.935199 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:58.935202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:58.935629 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:58.935630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:58.935754 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:58.936332 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:37:58.936341 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:37:58.937917 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:37:58.939803 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:37:58.939811 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:37:58.939814 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:37:58.940053 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:37:58.940272 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:37:58.940273 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:37:58.940274 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:37:58.940392 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:37:58.940668 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:37:58.940676 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:37:58.940819 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 14:37:58.940934 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 14:37:58.941094 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":115,"threads_started":1}
I20260811 14:37:58.942269 23360 heartbeater.cc:347] Connected to a master server at 127.2.103.126:36973
I20260811 14:37:58.942436 23360 heartbeater.cc:464] Registering TS with master...
I20260811 14:37:58.942726 23360 heartbeater.cc:511] Master 127.2.103.126:36973 requested a full tablet report, sending...
I20260811 14:37:58.943150 22621 ts_manager.cc:194] Registered new tserver with Master: c64d600ac05f484b88d41fe8dab2facb (127.2.103.67:36309)
I20260811 14:37:58.943614 22621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:36331
I20260811 14:37:58.949445 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:37:58.950121 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 22728
I20260811 14:37:59.010192 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 22949
Aug 11 14:37:59 dist-test-slave-twwt krb5kdc[21917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459078, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
I20260811 14:37:59.051014 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 23170
I20260811 14:37:59.069693 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 22571
I20260811 14:37:59.082129 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:36973
--webserver_interface=127.2.103.126
--webserver_port=42749
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:41313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:36973
--ranger_config_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.103.84:43053/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.103.84:43053/kms with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:37:59.173682 23363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:37:59.173934 23363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:37:59.173959 23363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:37:59.175212 23363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:37:59.175259 23363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:37:59.175274 23363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:37:59.175289 23363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:37:59.175302 23363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:37:59.177002 23363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:41313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.103.84:43053/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:36973
--ranger_config_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:36973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=42749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:37:59.177299 23363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:37:59.177563 23363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:37:59.182869 23373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:37:59.183060 23363 server_base.cc:1034] running on GCE node
W20260811 14:37:59.182870 23374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:37:59.183171 23378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:37:59.183449 23363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:37:59.183802 23363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:37:59.184985 23363 hybrid_clock.cc:648] HybridClock initialized: now 1786459079184946 us; error 55 us; skew 500 ppm
Aug 11 14:37:59 dist-test-slave-twwt krb5kdc[21917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459079, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:37:59.188154 23363 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:37:59.188696 23363 webserver.cc:460] Webserver started at http://127.2.103.126:42749/ using document root <none> and password file <none>
I20260811 14:37:59.188956 23363 fs_manager.cc:359] Metadata directory not provided
I20260811 14:37:59.189013 23363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 14:37:59 dist-test-slave-twwt krb5kdc[21917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459079, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
I20260811 14:37:59.270442 23363 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:37:59.271322 23391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:59.271497 23363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:37:59.271617 23363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "8297799e63e7413db1865893811513f2"
format_stamp: "Formatted at 2026-08-11 14:37:57 on dist-test-slave-twwt"
server_key: "e7140a63fc69bc6ec61f76b9455c9101"
server_key_iv: "ce131e0edbe47ac7c734d15e5250d42d"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:59.272010 23363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:37:59.291574 23363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:37:59.293959 23363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:37:59.294098 23363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:37:59.299077 23363 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:36973
I20260811 14:37:59.299115 23470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:36973 every 8 connection(s)
I20260811 14:37:59.299618 23363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:37:59.301383 23472 sys_catalog.cc:263] Verifying existing consensus state
I20260811 14:37:59.301860 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 23363
I20260811 14:37:59.302125 23472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2: Bootstrap starting.
I20260811 14:37:59.302189 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.103.84:43053/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:37127
--local_ip_for_outbound_sockets=127.2.103.65
--tserver_master_addrs=127.2.103.126:36973
--webserver_port=42091
--webserver_interface=127.2.103.65
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:41313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.103.84:43053/kms with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:37:59.310937 23472 log.cc:824] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2: Log is configured to *not* fsync() on all Append() calls
I20260811 14:37:59.312877 23472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:37:59.313195 23472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2: Bootstrap complete.
I20260811 14:37:59.315830 23472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8297799e63e7413db1865893811513f2" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36973 } }
I20260811 14:37:59.316109 23472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8297799e63e7413db1865893811513f2, State: Initialized, Role: FOLLOWER
I20260811 14:37:59.316308 23472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8297799e63e7413db1865893811513f2" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36973 } }
I20260811 14:37:59.316390 23472 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:37:59.316445 23472 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:37:59.316494 23472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [term 1 FOLLOWER]: Advancing to term 2
I20260811 14:37:59.317575 23472 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8297799e63e7413db1865893811513f2" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36973 } }
I20260811 14:37:59.317708 23472 leader_election.cc:302] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8297799e63e7413db1865893811513f2; no voters:
I20260811 14:37:59.317921 23472 leader_election.cc:288] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 14:37:59.317996 23478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 14:37:59.318174 23478 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [term 2 LEADER]: Becoming Leader. State: Replica: 8297799e63e7413db1865893811513f2, State: Running, Role: LEADER
I20260811 14:37:59.318228 23472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:37:59.318300 23478 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8297799e63e7413db1865893811513f2" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36973 } }
I20260811 14:37:59.318748 23480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8297799e63e7413db1865893811513f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8297799e63e7413db1865893811513f2" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36973 } } }
I20260811 14:37:59.318848 23480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [sys.catalog]: This master's current role is: LEADER
I20260811 14:37:59.319095 23483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8297799e63e7413db1865893811513f2. Latest consensus state: current_term: 2 leader_uuid: "8297799e63e7413db1865893811513f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8297799e63e7413db1865893811513f2" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36973 } } }
I20260811 14:37:59.319212 23483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2 [sys.catalog]: This master's current role is: LEADER
I20260811 14:37:59.319321 23494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:37:59.319837 23494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:37:59.319969 23494 catalog_manager.cc:1259] Loaded cluster ID: 118a9a9241c44cda986b352d758c6e11
I20260811 14:37:59.320001 23494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:37:59.320297 23472 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 14:37:59.321241 23494 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:37:59.321436 23494 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2: Loaded TSK: 0
I20260811 14:37:59.321650 23494 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 14:37:59.415943 23475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:37:59.416249 23475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:37:59.416287 23475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:37:59.418080 23475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:37:59.418149 23475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:37:59.418195 23475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:37:59.420636 23475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:41313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.103.84:43053/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:37127
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=42091
--tserver_master_addrs=127.2.103.126:36973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:37:59.421002 23475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:37:59.421329 23475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:37:59.428550 23537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:37:59.428550 23538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:37:59.428668 23540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:37:59.429003 23475 server_base.cc:1034] running on GCE node
I20260811 14:37:59.429255 23475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:37:59.429601 23475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:37:59.430792 23475 hybrid_clock.cc:648] HybridClock initialized: now 1786459079430757 us; error 47 us; skew 500 ppm
Aug 11 14:37:59 dist-test-slave-twwt krb5kdc[21917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459079, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:37:59.434636 23475 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:37:59.435263 23475 webserver.cc:460] Webserver started at http://127.2.103.65:42091/ using document root <none> and password file <none>
I20260811 14:37:59.435570 23475 fs_manager.cc:359] Metadata directory not provided
I20260811 14:37:59.435665 23475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 14:37:59 dist-test-slave-twwt krb5kdc[21917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459079, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
I20260811 14:37:59.461007 23475 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260811 14:37:59.462373 23552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:59.462561 23475 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:37:59.462708 23475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "1c7a8e78377d4355a9568dd688d9aa91"
format_stamp: "Formatted at 2026-08-11 14:37:58 on dist-test-slave-twwt"
server_key: "e61c9faf165c97b7be0cfda68ee88c7a"
server_key_iv: "856dfe821aa0434818ab9490d817474d"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:59.463229 23475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:37:59.489923 23475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:37:59.492570 23475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:37:59.492811 23475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:37:59.493095 23475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:37:59.493500 23475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:37:59.493554 23475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:59.493587 23475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:37:59.493613 23475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:59.500608 23475 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:37127
I20260811 14:37:59.500806 23735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:37127 every 8 connection(s)
I20260811 14:37:59.501190 23475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:37:59.509863 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 23475
I20260811 14:37:59.510218 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.103.84:43053/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:34585
--local_ip_for_outbound_sockets=127.2.103.66
--tserver_master_addrs=127.2.103.126:36973
--webserver_port=35965
--webserver_interface=127.2.103.66
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:41313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.103.84:43053/kms with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 14:37:59 dist-test-slave-twwt krb5kdc[21917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459079, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:37:59.515123 23739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:59.502088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:36973 (local address 127.2.103.65:52797)
0811 14:37:59.502315 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 14:37:59.502366 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 14:37:59.503016 (+ 650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:37:59.503644 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:37:59.503648 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:37:59.503933 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:37:59.504180 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:59.504186 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:59.506147 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:59.506148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:59.507472 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:59.507476 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:59.508510 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:59.508512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:59.508614 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:59.509206 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:37:59.509215 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:37:59.510864 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:37:59.512947 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:37:59.512954 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:37:59.512957 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:37:59.513291 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:37:59.513583 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:37:59.513584 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:37:59.513584 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:37:59.513715 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:37:59.514340 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:37:59.514348 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:37:59.514520 (+ 172us) client_negotiation.cc:769] Sending connection context
0811 14:37:59.514658 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 14:37:59.514859 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":104,"threads_started":1}
I20260811 14:37:59.515163 23742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:59.502540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:52797 (local address 127.2.103.126:36973)
0811 14:37:59.502794 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 14:37:59.502805 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:37:59.502857 (+ 52us) server_negotiation.cc:407] Connection header received
0811 14:37:59.503296 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:37:59.503309 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:37:59.503384 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:37:59.503559 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:37:59.504323 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:59.506010 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:59.507622 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:59.508368 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:59.508500 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:59.511008 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:37:59.511013 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:37:59.511013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:37:59.511024 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:37:59.512835 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:59.513392 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:59.513395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:59.513396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:59.513509 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:59.513827 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:59.513829 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:59.513830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:59.514180 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:37:59.514302 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 14:37:59.514763 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 14:37:59.514912 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":124,"threads_started":1}
I20260811 14:37:59.516359 23737 heartbeater.cc:347] Connected to a master server at 127.2.103.126:36973
I20260811 14:37:59.516479 23737 heartbeater.cc:464] Registering TS with master...
I20260811 14:37:59.516701 23737 heartbeater.cc:511] Master 127.2.103.126:36973 requested a full tablet report, sending...
W20260811 14:37:59.517752 23737 heartbeater.cc:644] Failed to heartbeat to 127.2.103.126:36973 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 14:37:59.517850 23737 heartbeater.cc:464] Registering TS with master...
I20260811 14:37:59.517928 23737 heartbeater.cc:511] Master 127.2.103.126:36973 requested a full tablet report, sending...
I20260811 14:37:59.518362 23737 heartbeater.cc:464] Registering TS with master...
I20260811 14:37:59.518464 23737 heartbeater.cc:511] Master 127.2.103.126:36973 requested a full tablet report, sending...
W20260811 14:37:59.518996 23737 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 14:37:59.633592 23743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:37:59.633893 23743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:37:59.633936 23743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:37:59.635757 23743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:37:59.635828 23743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:37:59.635870 23743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:37:59.638154 23743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:41313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.103.84:43053/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:34585
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=35965
--tserver_master_addrs=127.2.103.126:36973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:37:59.638453 23743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:37:59.638797 23743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 14:37:59.645496 23743 server_base.cc:1034] running on GCE node
W20260811 14:37:59.645614 23753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:37:59.647516 23750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:37:59.650332 23749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:37:59.650734 23743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:37:59.651072 23743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:37:59.652267 23743 hybrid_clock.cc:648] HybridClock initialized: now 1786459079652241 us; error 46 us; skew 500 ppm
Aug 11 14:37:59 dist-test-slave-twwt krb5kdc[21917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459079, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:37:59.655236 23743 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:37:59.655799 23743 webserver.cc:460] Webserver started at http://127.2.103.66:35965/ using document root <none> and password file <none>
I20260811 14:37:59.656067 23743 fs_manager.cc:359] Metadata directory not provided
I20260811 14:37:59.656157 23743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 14:37:59 dist-test-slave-twwt krb5kdc[21917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459079, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
I20260811 14:37:59.675156 23743 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.001s sys 0.000s
I20260811 14:37:59.676580 23765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:59.676764 23743 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:37:59.676879 23743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "619ce6b5451f48149031f2f7223a0fec"
format_stamp: "Formatted at 2026-08-11 14:37:58 on dist-test-slave-twwt"
server_key: "7ef193971188a266f8a0faecc6fd0037"
server_key_iv: "0d7dbfe75ab761c4d436cd4588d6d26c"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:59.677407 23743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:37:59.721463 23743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:37:59.724134 23743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:37:59.724335 23743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:37:59.724611 23743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:37:59.725067 23743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:37:59.725135 23743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:59.725172 23743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:37:59.725198 23743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:59.733021 23743 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:34585
I20260811 14:37:59.733565 23743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:37:59.735536 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 23743
I20260811 14:37:59.735839 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.103.84:43053/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:36309
--local_ip_for_outbound_sockets=127.2.103.67
--tserver_master_addrs=127.2.103.126:36973
--webserver_port=40559
--webserver_interface=127.2.103.67
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:41313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.103.84:43053/kms with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:37:59.739012 23937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:34585 every 8 connection(s)
Aug 11 14:37:59 dist-test-slave-twwt krb5kdc[21917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459079, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:37:59.744355 23742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:59.734780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:34495 (local address 127.2.103.126:36973)
0811 14:37:59.734827 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 14:37:59.734829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:37:59.734840 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:37:59.735444 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:37:59.735446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:37:59.735462 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:37:59.735496 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:37:59.736509 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:59.737299 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:59.738493 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:59.738829 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:59.738899 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:59.741298 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:37:59.741300 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:37:59.741300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:37:59.741308 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:37:59.742860 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:59.743322 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:59.743323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:59.743324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:59.743383 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:59.743661 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:59.743662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:59.743663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:59.743790 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:37:59.743866 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 14:37:59.744165 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 14:37:59.744258 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 14:37:59.744820 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:59.734489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:36973 (local address 127.2.103.66:34495)
0811 14:37:59.734676 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 14:37:59.734731 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 14:37:59.735255 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:37:59.735814 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:37:59.735819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:37:59.736081 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:37:59.736374 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:59.736381 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:59.737431 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:59.737432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:59.738378 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:59.738381 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:59.739004 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:59.739005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:59.739154 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:59.739774 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:37:59.739784 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:37:59.741153 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:37:59.742962 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:37:59.742970 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:37:59.742973 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:37:59.743225 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:37:59.743451 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:37:59.743452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:37:59.743452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:37:59.743583 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:37:59.743857 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:37:59.743865 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:37:59.744024 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 14:37:59.744142 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 14:37:59.744315 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260811 14:37:59.745412 23938 heartbeater.cc:347] Connected to a master server at 127.2.103.126:36973
I20260811 14:37:59.745517 23938 heartbeater.cc:464] Registering TS with master...
I20260811 14:37:59.745713 23938 heartbeater.cc:511] Master 127.2.103.126:36973 requested a full tablet report, sending...
W20260811 14:37:59.746358 23938 heartbeater.cc:644] Failed to heartbeat to 127.2.103.126:36973 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 14:37:59.746434 23938 heartbeater.cc:464] Registering TS with master...
I20260811 14:37:59.746513 23938 heartbeater.cc:511] Master 127.2.103.126:36973 requested a full tablet report, sending...
I20260811 14:37:59.746865 23938 heartbeater.cc:464] Registering TS with master...
I20260811 14:37:59.746946 23938 heartbeater.cc:511] Master 127.2.103.126:36973 requested a full tablet report, sending...
W20260811 14:37:59.747238 23938 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 14:37:59.866223 23944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:37:59.866561 23944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:37:59.866633 23944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:37:59.868402 23944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:37:59.868471 23944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:37:59.868515 23944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:37:59.870548 23944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:41313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.103.84:43053/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:36309
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=40559
--tserver_master_addrs=127.2.103.126:36973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:37:59.870836 23944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:37:59.871160 23944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:37:59.878165 23951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:37:59.878314 23944 server_base.cc:1034] running on GCE node
W20260811 14:37:59.878175 23952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:37:59.878533 23955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:37:59.878806 23944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:37:59.879141 23944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:37:59.880342 23944 hybrid_clock.cc:648] HybridClock initialized: now 1786459079880325 us; error 29 us; skew 500 ppm
Aug 11 14:37:59 dist-test-slave-twwt krb5kdc[21917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459079, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:37:59.883579 23944 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:37:59.884198 23944 webserver.cc:460] Webserver started at http://127.2.103.67:40559/ using document root <none> and password file <none>
I20260811 14:37:59.884483 23944 fs_manager.cc:359] Metadata directory not provided
I20260811 14:37:59.884588 23944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 14:37:59 dist-test-slave-twwt krb5kdc[21917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459079, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
I20260811 14:37:59.899421 23944 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260811 14:37:59.900527 23972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:59.900734 23944 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:37:59.900842 23944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "c64d600ac05f484b88d41fe8dab2facb"
format_stamp: "Formatted at 2026-08-11 14:37:58 on dist-test-slave-twwt"
server_key: "51ca9c0227ddf729647484940f9ccb0a"
server_key_iv: "56790d8d5df2b52363aec79856c0d224"
server_key_version: "kuduclusterkey@0"
I20260811 14:37:59.901273 23944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:37:59.913724 23944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:37:59.915966 23944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:37:59.916118 23944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:37:59.916405 23944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:37:59.916744 23944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:37:59.916805 23944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:59.916848 23944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:37:59.916874 23944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:37:59.923694 23944 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:36309
I20260811 14:37:59.923902 24175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:36309 every 8 connection(s)
I20260811 14:37:59.924160 23944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:37:59.924842 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 23944
Aug 11 14:37:59 dist-test-slave-twwt krb5kdc[21917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459079, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:37:59.936911 23742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:59.925450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:35475 (local address 127.2.103.126:36973)
0811 14:37:59.925497 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 14:37:59.925498 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:37:59.925512 (+ 14us) server_negotiation.cc:407] Connection header received
0811 14:37:59.926098 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:37:59.926099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:37:59.926109 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:37:59.926161 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:37:59.926920 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:59.927666 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:59.929724 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:59.930011 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:59.930085 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:59.933356 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:37:59.933358 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:37:59.933359 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:37:59.933367 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:37:59.935163 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:59.935726 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:59.935727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:59.935728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:59.935797 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:59.936105 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:59.936106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:59.936107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:59.936256 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:37:59.936303 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 14:37:59.936695 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 14:37:59.936797 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:37:59.937779 24179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:59.925102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:36973 (local address 127.2.103.67:35475)
0811 14:37:59.925320 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 14:37:59.925374 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 14:37:59.925996 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:37:59.926234 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:37:59.926238 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:37:59.926463 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:37:59.926776 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:59.926784 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:59.927796 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:59.927797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:59.929338 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:37:59.929342 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:37:59.930177 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:37:59.930179 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:37:59.930359 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:59.931411 (+ 1052us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:37:59.931422 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:37:59.933223 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:37:59.935325 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:37:59.935333 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:37:59.935336 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:37:59.935635 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:37:59.935879 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:37:59.935880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:37:59.935880 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:37:59.936021 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:37:59.936358 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:37:59.936365 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:37:59.936533 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 14:37:59.937324 (+ 791us) client_negotiation.cc:240] Negotiation successful
0811 14:37:59.937525 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":109,"threads_started":1}
I20260811 14:37:59.938427 24177 heartbeater.cc:347] Connected to a master server at 127.2.103.126:36973
I20260811 14:37:59.938516 24177 heartbeater.cc:464] Registering TS with master...
I20260811 14:37:59.938715 24177 heartbeater.cc:511] Master 127.2.103.126:36973 requested a full tablet report, sending...
W20260811 14:37:59.939438 24177 heartbeater.cc:644] Failed to heartbeat to 127.2.103.126:36973 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 14:37:59.939522 24177 heartbeater.cc:464] Registering TS with master...
I20260811 14:37:59.939589 24177 heartbeater.cc:511] Master 127.2.103.126:36973 requested a full tablet report, sending...
I20260811 14:37:59.939973 24177 heartbeater.cc:464] Registering TS with master...
I20260811 14:37:59.940049 24177 heartbeater.cc:511] Master 127.2.103.126:36973 requested a full tablet report, sending...
W20260811 14:37:59.940389 24177 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 14:37:59.941601 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:37:59.925495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51026 (local address 127.2.103.126:36973)
0811 14:37:59.925585 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 14:37:59.925587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:37:59.926851 (+ 1264us) server_negotiation.cc:407] Connection header received
0811 14:37:59.926925 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:37:59.926926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:37:59.926940 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:37:59.926996 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:37:59.927618 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:59.928111 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:59.928885 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:37:59.929098 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:37:59.929176 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:37:59.930667 (+ 1491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:37:59.930669 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:37:59.930669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:37:59.930676 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:37:59.940316 (+ 9640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:59.940776 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:59.940777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:59.940778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:59.940844 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:37:59.941047 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:37:59.941048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:37:59.941048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:37:59.941166 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:37:59.941359 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 14:37:59.941402 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 14:37:59.941489 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":57,"threads_started":1}
Aug 11 14:38:00 dist-test-slave-twwt krb5kdc[21917](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786459080, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:00.519192 23737 heartbeater.cc:464] Registering TS with master...
I20260811 14:38:00.519371 23737 heartbeater.cc:511] Master 127.2.103.126:36973 requested a full tablet report, sending...
I20260811 14:38:00.519848 23415 ts_manager.cc:194] Registered new tserver with Master: 1c7a8e78377d4355a9568dd688d9aa91 (127.2.103.65:37127)
I20260811 14:38:00.520931 23415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:52797
I20260811 14:38:00.747440 23938 heartbeater.cc:464] Registering TS with master...
I20260811 14:38:00.747737 23938 heartbeater.cc:511] Master 127.2.103.126:36973 requested a full tablet report, sending...
I20260811 14:38:00.748136 23415 ts_manager.cc:194] Registered new tserver with Master: 619ce6b5451f48149031f2f7223a0fec (127.2.103.66:34585)
I20260811 14:38:00.748668 23415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:34495
Aug 11 14:38:00 dist-test-slave-twwt krb5kdc[21917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459080, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for HTTP/127.2.103.84@KRBTEST.COM
I20260811 14:38:00.940562 24177 heartbeater.cc:464] Registering TS with master...
I20260811 14:38:00.940757 24177 heartbeater.cc:511] Master 127.2.103.126:36973 requested a full tablet report, sending...
I20260811 14:38:00.941141 23415 ts_manager.cc:194] Registered new tserver with Master: c64d600ac05f484b88d41fe8dab2facb (127.2.103.67:36309)
I20260811 14:38:00.941689 23415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:35475
I20260811 14:38:00.943274 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:38:00.959403 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:00.944311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51032 (local address 127.2.103.126:36973)
0811 14:38:00.944584 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:00.944587 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:00.950892 (+ 6305us) server_negotiation.cc:407] Connection header received
0811 14:38:00.950961 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:00.950963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:00.950979 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:00.951033 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:00.951918 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:00.952691 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:00.953809 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:00.954117 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:00.954162 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:00.955893 (+ 1731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:00.955894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:00.955895 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:00.955903 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:00.957866 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:00.958306 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:00.958308 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:00.958308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:00.958375 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:00.958595 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:00.958596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:00.958597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:00.958793 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:00.958963 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:00.959197 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 14:38:00.959294 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":82,"threads_started":1}
I20260811 14:38:00.962106 23415 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51032:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:38:00.965106 23415 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:38:00.992132 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:00.979030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:34585 (local address 127.0.0.1:41050)
0811 14:38:00.982770 (+ 3740us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:00.982832 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:00.982947 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:00.983798 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:00.983802 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:00.983805 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:00.983894 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:00.983896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:00.985586 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:00.985587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:00.986976 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:00.986980 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:00.991772 (+ 4792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:00.991773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:00.991880 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:00.991895 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 14:38:00.991966 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 14:38:00.992031 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3684,"mutex_wait_us":2,"thread_start_us":81,"threads_started":1}
I20260811 14:38:00.993628 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:00.979280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:37127 (local address 127.0.0.1:38436)
0811 14:38:00.983076 (+ 3796us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:00.983085 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:00.983163 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:00.987760 (+ 4597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:00.987761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:00.987764 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:00.987823 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:00.987824 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:00.989082 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:00.989083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:00.990245 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:00.990248 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:00.993379 (+ 3131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:00.993380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:00.993460 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:00.993463 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:38:00.993511 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 14:38:00.993551 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3779,"thread_start_us":69,"threads_started":1}
I20260811 14:38:00.993968 24229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:00.979852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41050 (local address 127.2.103.66:34585)
0811 14:38:00.983383 (+ 3531us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:00.983389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:00.983401 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:38:00.983441 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:00.983444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:00.983461 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:00.983664 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:00.983978 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:00.985491 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:00.990358 (+ 4867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:00.991557 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:00.991639 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:00.991702 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:00.993836 (+ 2134us) server_negotiation.cc:299] Negotiation successful
0811 14:38:00.993887 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3499,"thread_start_us":64,"threads_started":1}
I20260811 14:38:00.994445 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:00.980001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38436 (local address 127.2.103.65:37127)
0811 14:38:00.987142 (+ 7141us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:00.987148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:00.987160 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:38:00.987200 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:00.987203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:00.987222 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:00.987643 (+ 421us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:00.987900 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:00.988983 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:00.992386 (+ 3403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:00.993202 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:00.993283 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:00.993340 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:00.994318 (+ 978us) server_negotiation.cc:299] Negotiation successful
0811 14:38:00.994366 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7092,"thread_start_us":52,"threads_started":1}
I20260811 14:38:00.994802 23838 tablet_service.cc:1467] Processing CreateTablet for tablet 23ebe64c4d534d7e899e730b47436013 (DEFAULT_TABLE table=test-table [id=a2772464012a4ce99db5f22f1988cb6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:38:00.995148 23623 tablet_service.cc:1467] Processing CreateTablet for tablet 23ebe64c4d534d7e899e730b47436013 (DEFAULT_TABLE table=test-table [id=a2772464012a4ce99db5f22f1988cb6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:38:00.995271 23838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23ebe64c4d534d7e899e730b47436013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:00.995509 23623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23ebe64c4d534d7e899e730b47436013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:00.999408 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:00.979444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:36309 (local address 127.0.0.1:50412)
0811 14:38:00.983242 (+ 3798us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:00.983251 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:00.983284 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:00.990383 (+ 7099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:00.990385 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:00.990388 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:00.990456 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:00.990457 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:00.991828 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:00.991829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:00.993086 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:00.993089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:00.994118 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:00.994119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:00.994195 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:00.994199 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:38:00.999236 (+ 5037us) client_negotiation.cc:240] Negotiation successful
0811 14:38:00.999320 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3765,"thread_start_us":76,"threads_started":1}
I20260811 14:38:00.999679 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:00.980138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50412 (local address 127.2.103.67:36309)
0811 14:38:00.990065 (+ 9927us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:00.990070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:00.990087 (+ 17us) server_negotiation.cc:407] Connection header received
0811 14:38:00.990129 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:00.990131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:00.990148 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:00.990266 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:00.990543 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:00.991677 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:00.993199 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:00.993959 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:00.994032 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:00.994081 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:00.999546 (+ 5465us) server_negotiation.cc:299] Negotiation successful
0811 14:38:00.999602 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9882,"thread_start_us":57,"threads_started":1}
I20260811 14:38:01.000437 24062 tablet_service.cc:1467] Processing CreateTablet for tablet 23ebe64c4d534d7e899e730b47436013 (DEFAULT_TABLE table=test-table [id=a2772464012a4ce99db5f22f1988cb6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:38:01.000845 24062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23ebe64c4d534d7e899e730b47436013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:01.006839 24235 tablet_bootstrap.cc:492] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec: Bootstrap starting.
I20260811 14:38:01.007607 24235 tablet_bootstrap.cc:654] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:01.015427 24236 tablet_bootstrap.cc:492] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb: Bootstrap starting.
I20260811 14:38:01.016408 24236 tablet_bootstrap.cc:654] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:01.016799 24236 log.cc:824] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:01.017262 24234 tablet_bootstrap.cc:492] T 23ebe64c4d534d7e899e730b47436013 P 1c7a8e78377d4355a9568dd688d9aa91: Bootstrap starting.
I20260811 14:38:01.018023 24234 tablet_bootstrap.cc:654] T 23ebe64c4d534d7e899e730b47436013 P 1c7a8e78377d4355a9568dd688d9aa91: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:01.018369 24234 log.cc:824] T 23ebe64c4d534d7e899e730b47436013 P 1c7a8e78377d4355a9568dd688d9aa91: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:01.018924 24235 log.cc:824] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:01.019924 24236 tablet_bootstrap.cc:492] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb: No bootstrap required, opened a new log
I20260811 14:38:01.020009 24236 ts_tablet_manager.cc:1397] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 14:38:01.022976 24236 raft_consensus.cc:348] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619ce6b5451f48149031f2f7223a0fec" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 34585 } } peers { permanent_uuid: "1c7a8e78377d4355a9568dd688d9aa91" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 37127 } } peers { permanent_uuid: "c64d600ac05f484b88d41fe8dab2facb" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36309 } }
I20260811 14:38:01.023186 24236 raft_consensus.cc:374] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:01.023226 24236 raft_consensus.cc:729] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c64d600ac05f484b88d41fe8dab2facb, State: Initialized, Role: FOLLOWER
I20260811 14:38:01.023357 24236 consensus_queue.cc:260] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619ce6b5451f48149031f2f7223a0fec" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 34585 } } peers { permanent_uuid: "1c7a8e78377d4355a9568dd688d9aa91" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 37127 } } peers { permanent_uuid: "c64d600ac05f484b88d41fe8dab2facb" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36309 } }
I20260811 14:38:01.023669 24236 ts_tablet_manager.cc:1428] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 14:38:01.023825 24177 heartbeater.cc:503] Master 127.2.103.126:36973 was elected leader, sending a full tablet report...
I20260811 14:38:01.025177 24234 tablet_bootstrap.cc:492] T 23ebe64c4d534d7e899e730b47436013 P 1c7a8e78377d4355a9568dd688d9aa91: No bootstrap required, opened a new log
I20260811 14:38:01.025264 24234 ts_tablet_manager.cc:1397] T 23ebe64c4d534d7e899e730b47436013 P 1c7a8e78377d4355a9568dd688d9aa91: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20260811 14:38:01.028021 24234 raft_consensus.cc:348] T 23ebe64c4d534d7e899e730b47436013 P 1c7a8e78377d4355a9568dd688d9aa91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619ce6b5451f48149031f2f7223a0fec" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 34585 } } peers { permanent_uuid: "1c7a8e78377d4355a9568dd688d9aa91" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 37127 } } peers { permanent_uuid: "c64d600ac05f484b88d41fe8dab2facb" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36309 } }
I20260811 14:38:01.028225 24234 raft_consensus.cc:374] T 23ebe64c4d534d7e899e730b47436013 P 1c7a8e78377d4355a9568dd688d9aa91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:01.028272 24234 raft_consensus.cc:729] T 23ebe64c4d534d7e899e730b47436013 P 1c7a8e78377d4355a9568dd688d9aa91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7a8e78377d4355a9568dd688d9aa91, State: Initialized, Role: FOLLOWER
I20260811 14:38:01.028403 24234 consensus_queue.cc:260] T 23ebe64c4d534d7e899e730b47436013 P 1c7a8e78377d4355a9568dd688d9aa91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619ce6b5451f48149031f2f7223a0fec" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 34585 } } peers { permanent_uuid: "1c7a8e78377d4355a9568dd688d9aa91" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 37127 } } peers { permanent_uuid: "c64d600ac05f484b88d41fe8dab2facb" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36309 } }
I20260811 14:38:01.028710 24234 ts_tablet_manager.cc:1428] T 23ebe64c4d534d7e899e730b47436013 P 1c7a8e78377d4355a9568dd688d9aa91: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260811 14:38:01.028855 23737 heartbeater.cc:503] Master 127.2.103.126:36973 was elected leader, sending a full tablet report...
I20260811 14:38:01.030414 24235 tablet_bootstrap.cc:492] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec: No bootstrap required, opened a new log
I20260811 14:38:01.030508 24235 ts_tablet_manager.cc:1397] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20260811 14:38:01.032770 24235 raft_consensus.cc:348] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619ce6b5451f48149031f2f7223a0fec" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 34585 } } peers { permanent_uuid: "1c7a8e78377d4355a9568dd688d9aa91" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 37127 } } peers { permanent_uuid: "c64d600ac05f484b88d41fe8dab2facb" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36309 } }
I20260811 14:38:01.032964 24235 raft_consensus.cc:374] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:01.033003 24235 raft_consensus.cc:729] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 619ce6b5451f48149031f2f7223a0fec, State: Initialized, Role: FOLLOWER
I20260811 14:38:01.033124 24235 consensus_queue.cc:260] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619ce6b5451f48149031f2f7223a0fec" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 34585 } } peers { permanent_uuid: "1c7a8e78377d4355a9568dd688d9aa91" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 37127 } } peers { permanent_uuid: "c64d600ac05f484b88d41fe8dab2facb" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36309 } }
I20260811 14:38:01.033411 24235 ts_tablet_manager.cc:1428] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:38:01.033532 23938 heartbeater.cc:503] Master 127.2.103.126:36973 was elected leader, sending a full tablet report...
I20260811 14:38:01.067893 24246 raft_consensus.cc:482] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:38:01.068063 24246 raft_consensus.cc:504] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619ce6b5451f48149031f2f7223a0fec" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 34585 } } peers { permanent_uuid: "1c7a8e78377d4355a9568dd688d9aa91" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 37127 } } peers { permanent_uuid: "c64d600ac05f484b88d41fe8dab2facb" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36309 } }
I20260811 14:38:01.068486 24246 leader_election.cc:288] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c7a8e78377d4355a9568dd688d9aa91 (127.2.103.65:37127), c64d600ac05f484b88d41fe8dab2facb (127.2.103.67:36309)
I20260811 14:38:01.073504 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:01.068981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:43237 (local address 127.2.103.65:37127)
0811 14:38:01.069078 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:01.069081 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:01.069438 (+ 357us) server_negotiation.cc:407] Connection header received
0811 14:38:01.069664 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:01.069665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:01.069676 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:01.069706 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:01.070344 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:01.071075 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:01.072533 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:01.073098 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:01.073145 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:01.073154 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:01.073386 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 14:38:01.073426 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:38:01.073800 24247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:01.068741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:37127 (local address 127.2.103.66:43237)
0811 14:38:01.069364 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:01.069375 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:01.069446 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:01.070190 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:01.070192 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:01.070194 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:01.070263 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:01.070264 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:01.071169 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:01.071170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:01.072430 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:01.072433 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:01.073236 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:01.073237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:01.073306 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:01.073310 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:38:01.073655 (+ 345us) client_negotiation.cc:240] Negotiation successful
0811 14:38:01.073700 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":128,"threads_started":1}
I20260811 14:38:01.074162 23662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23ebe64c4d534d7e899e730b47436013" candidate_uuid: "619ce6b5451f48149031f2f7223a0fec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7a8e78377d4355a9568dd688d9aa91" is_pre_election: true
I20260811 14:38:01.074373 23662 raft_consensus.cc:2454] T 23ebe64c4d534d7e899e730b47436013 P 1c7a8e78377d4355a9568dd688d9aa91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 619ce6b5451f48149031f2f7223a0fec in term 0.
I20260811 14:38:01.074682 23766 leader_election.cc:302] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7a8e78377d4355a9568dd688d9aa91, 619ce6b5451f48149031f2f7223a0fec; no voters:
I20260811 14:38:01.074838 24246 raft_consensus.cc:2781] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:38:01.074935 24246 raft_consensus.cc:482] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:38:01.074970 24246 raft_consensus.cc:3037] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:38:01.075937 24246 raft_consensus.cc:504] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619ce6b5451f48149031f2f7223a0fec" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 34585 } } peers { permanent_uuid: "1c7a8e78377d4355a9568dd688d9aa91" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 37127 } } peers { permanent_uuid: "c64d600ac05f484b88d41fe8dab2facb" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36309 } }
I20260811 14:38:01.076087 24246 leader_election.cc:288] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [CANDIDATE]: Term 1 election: Requested vote from peers 1c7a8e78377d4355a9568dd688d9aa91 (127.2.103.65:37127), c64d600ac05f484b88d41fe8dab2facb (127.2.103.67:36309)
I20260811 14:38:01.076339 23662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23ebe64c4d534d7e899e730b47436013" candidate_uuid: "619ce6b5451f48149031f2f7223a0fec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7a8e78377d4355a9568dd688d9aa91"
I20260811 14:38:01.076428 23662 raft_consensus.cc:3037] T 23ebe64c4d534d7e899e730b47436013 P 1c7a8e78377d4355a9568dd688d9aa91 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:38:01.077345 23662 raft_consensus.cc:2454] T 23ebe64c4d534d7e899e730b47436013 P 1c7a8e78377d4355a9568dd688d9aa91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 619ce6b5451f48149031f2f7223a0fec in term 1.
I20260811 14:38:01.077534 23766 leader_election.cc:302] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7a8e78377d4355a9568dd688d9aa91, 619ce6b5451f48149031f2f7223a0fec; no voters:
I20260811 14:38:01.077637 24246 raft_consensus.cc:2781] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:38:01.077800 24246 raft_consensus.cc:686] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [term 1 LEADER]: Becoming Leader. State: Replica: 619ce6b5451f48149031f2f7223a0fec, State: Running, Role: LEADER
I20260811 14:38:01.077903 24246 consensus_queue.cc:237] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619ce6b5451f48149031f2f7223a0fec" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 34585 } } peers { permanent_uuid: "1c7a8e78377d4355a9568dd688d9aa91" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 37127 } } peers { permanent_uuid: "c64d600ac05f484b88d41fe8dab2facb" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36309 } }
I20260811 14:38:01.078805 23415 catalog_manager.cc:5495] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec reported cstate change: term changed from 0 to 1, leader changed from <none> to 619ce6b5451f48149031f2f7223a0fec (127.2.103.66). New cstate: current_term: 1 leader_uuid: "619ce6b5451f48149031f2f7223a0fec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619ce6b5451f48149031f2f7223a0fec" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 34585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c7a8e78377d4355a9568dd688d9aa91" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 37127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c64d600ac05f484b88d41fe8dab2facb" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36309 } health_report { overall_health: UNKNOWN } } }
I20260811 14:38:01.079527 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:01.069270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:47765 (local address 127.2.103.67:36309)
0811 14:38:01.069328 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:01.069330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:01.069565 (+ 235us) server_negotiation.cc:407] Connection header received
0811 14:38:01.069632 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:01.069633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:01.069643 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:01.069697 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:01.070887 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:01.071566 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:01.074088 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:01.074650 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:01.074705 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:01.074714 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:01.079392 (+ 4678us) server_negotiation.cc:299] Negotiation successful
0811 14:38:01.079457 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 14:38:01.080029 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:01.069109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:36309 (local address 127.2.103.66:47765)
0811 14:38:01.069523 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:01.069530 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:01.069574 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:01.069753 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:01.069754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:01.069756 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:01.069842 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:01.069845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:01.071669 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:01.071669 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:01.072733 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:01.072736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:01.079193 (+ 6457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:01.079194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:01.079283 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:01.079289 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 14:38:01.079899 (+ 610us) client_negotiation.cc:240] Negotiation successful
0811 14:38:01.079951 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":66,"threads_started":1}
I20260811 14:38:01.080365 24097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23ebe64c4d534d7e899e730b47436013" candidate_uuid: "619ce6b5451f48149031f2f7223a0fec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c64d600ac05f484b88d41fe8dab2facb" is_pre_election: true
I20260811 14:38:01.080543 24097 raft_consensus.cc:2454] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 619ce6b5451f48149031f2f7223a0fec in term 0.
I20260811 14:38:01.080363 24094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23ebe64c4d534d7e899e730b47436013" candidate_uuid: "619ce6b5451f48149031f2f7223a0fec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c64d600ac05f484b88d41fe8dab2facb"
I20260811 14:38:01.080753 24094 raft_consensus.cc:3037] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:38:01.081611 24094 raft_consensus.cc:2454] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 619ce6b5451f48149031f2f7223a0fec in term 1.
I20260811 14:38:01.094153 24229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:01.090811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41066 (local address 127.2.103.66:34585)
0811 14:38:01.090854 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:01.090856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:01.090863 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:38:01.090904 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:01.090905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:01.090926 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:01.090990 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:38:01.091168 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:01.091893 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:01.093043 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:01.093353 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:01.093415 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:01.093613 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:38:01.093748 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:38:01.093823 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:01.094050 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 14:38:01.094073 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:38:01.097729 24094 raft_consensus.cc:1261] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb [term 1 FOLLOWER]: Refusing update from remote peer 619ce6b5451f48149031f2f7223a0fec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:38:01.098191 24249 consensus_queue.cc:1035] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [LEADER]: Connected to new peer: Peer: permanent_uuid: "c64d600ac05f484b88d41fe8dab2facb" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:38:01.100634 23662 raft_consensus.cc:1261] T 23ebe64c4d534d7e899e730b47436013 P 1c7a8e78377d4355a9568dd688d9aa91 [term 1 FOLLOWER]: Refusing update from remote peer 619ce6b5451f48149031f2f7223a0fec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:38:01.101009 24249 consensus_queue.cc:1035] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c7a8e78377d4355a9568dd688d9aa91" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 37127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:38:01.105953 24252 mvcc.cc:205] Tried to move back new op lower bound from 7317336396172738560 to 7317336396095528960. Current Snapshot: MvccSnapshot[applied={T|T < 7317336396172738560}]
I20260811 14:38:01.106283 24256 mvcc.cc:205] Tried to move back new op lower bound from 7317336396172738560 to 7317336396095528960. Current Snapshot: MvccSnapshot[applied={T|T < 7317336396172738560}]
I20260811 14:38:01.108769 24253 mvcc.cc:205] Tried to move back new op lower bound from 7317336396172738560 to 7317336396095528960. Current Snapshot: MvccSnapshot[applied={T|T < 7317336396172738560}]
I20260811 14:38:01.111811 23415 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:38:01.111913 23415 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:38:01.112727 23415 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8297799e63e7413db1865893811513f2: Sending DeleteTablet for 3 replicas of tablet 23ebe64c4d534d7e899e730b47436013
I20260811 14:38:01.113492 23838 tablet_service.cc:1514] Processing DeleteTablet for tablet 23ebe64c4d534d7e899e730b47436013 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:38:01 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:41050
I20260811 14:38:01.113747 23623 tablet_service.cc:1514] Processing DeleteTablet for tablet 23ebe64c4d534d7e899e730b47436013 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:38:01 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:38436
I20260811 14:38:01.114094 24062 tablet_service.cc:1514] Processing DeleteTablet for tablet 23ebe64c4d534d7e899e730b47436013 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:38:01 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:50412
I20260811 14:38:01.115373 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 23475
I20260811 14:38:01.120057 24268 tablet_replica.cc:330] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec: stopping tablet replica
I20260811 14:38:01.120225 24268 raft_consensus.cc:2229] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:38:01.120407 24268 raft_consensus.cc:2258] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:38:01.132493 24268 ts_tablet_manager.cc:1905] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:38:01.134439 24271 tablet_replica.cc:330] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb: stopping tablet replica
I20260811 14:38:01.134598 24271 raft_consensus.cc:2229] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:38:01.134847 24271 raft_consensus.cc:2258] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:38:01.135409 24271 ts_tablet_manager.cc:1905] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:38:01.137141 24271 ts_tablet_manager.cc:1918] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:38:01.137231 24271 log.cc:1196] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/23ebe64c4d534d7e899e730b47436013
I20260811 14:38:01.137529 24271 ts_tablet_manager.cc:1939] T 23ebe64c4d534d7e899e730b47436013 P c64d600ac05f484b88d41fe8dab2facb: Deleting consensus metadata
I20260811 14:38:01.138077 23396 catalog_manager.cc:4841] TS c64d600ac05f484b88d41fe8dab2facb (127.2.103.67:36309): tablet 23ebe64c4d534d7e899e730b47436013 (table test-table [id=a2772464012a4ce99db5f22f1988cb6c]) successfully deleted
W20260811 14:38:01.138361 23392 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.103.65:37127 (error 108)
I20260811 14:38:01.138751 24268 ts_tablet_manager.cc:1918] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:38:01.138857 24268 log.cc:1196] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/23ebe64c4d534d7e899e730b47436013
I20260811 14:38:01.139200 24268 ts_tablet_manager.cc:1939] T 23ebe64c4d534d7e899e730b47436013 P 619ce6b5451f48149031f2f7223a0fec: Deleting consensus metadata
I20260811 14:38:01.139689 23394 catalog_manager.cc:4841] TS 619ce6b5451f48149031f2f7223a0fec (127.2.103.66:34585): tablet 23ebe64c4d534d7e899e730b47436013 (table test-table [id=a2772464012a4ce99db5f22f1988cb6c]) successfully deleted
I20260811 14:38:01.146764 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 23743
I20260811 14:38:01.195453 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:01.195260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:37127 (local address 127.0.0.1:38444)
0811 14:38:01.195325 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:01.195398 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.65:37127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 14:38:01.195642 23392 catalog_manager.cc:4584] TS 1c7a8e78377d4355a9568dd688d9aa91 (127.2.103.65:37127): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 23ebe64c4d534d7e899e730b47436013: Network error: Client connection negotiation failed: client connection to 127.2.103.65:37127: connect: Connection refused (error 111)
I20260811 14:38:01.201028 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 23944
I20260811 14:38:01.216179 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 23363
I20260811 14:38:01.235363 2461 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 14:38:01.580549 2461 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 14:38:01.580685 2461 mini_ranger.cc:66] Stopping Ranger...
I20260811 14:38:01.936086 2461 mini_ranger.cc:68] Stopped Ranger
2026-08-11 14:38:01.936 UTC [21961] LOG: received smart shutdown request
2026-08-11 14:38:01.941 UTC [21961] LOG: background worker "logical replication launcher" (PID 21972) exited with exit code 1
2026-08-11 14:38:01.941 UTC [21965] LOG: shutting down
2026-08-11 14:38:02.125 UTC [21961] LOG: database system is shut down
2026-08-11T14:38:02Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50620 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:38:02 dist-test-slave-twwt krb5kdc[24289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:38:02 dist-test-slave-twwt krb5kdc[24289](info): set up 2 sockets
Aug 11 14:38:02 dist-test-slave-twwt krb5kdc[24289](info): commencing operation
krb5kdc: starting...
W20260811 14:38:03.514072 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:38:03 dist-test-slave-twwt krb5kdc[24289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:38:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:38:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:38:03.649643 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:40961
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:43721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:40961,127.2.103.125:43191,127.2.103.124:33343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:38:03.735610 24315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:03.735833 24315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:03.735857 24315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:03.737042 24315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:38:03.737087 24315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:03.737100 24315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:03.737113 24315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:38:03.737125 24315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:38:03.738732 24315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:43721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:40961,127.2.103.125:43191,127.2.103.124:33343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:40961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:03.739012 24315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:03.739287 24315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:03.744728 24323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:03.744728 24322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:03.744768 24326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:03.745061 24315 server_base.cc:1034] running on GCE node
I20260811 14:38:03.745286 24315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:03.745572 24315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:03.746688 24315 hybrid_clock.cc:648] HybridClock initialized: now 1786459083746633 us; error 63 us; skew 500 ppm
Aug 11 14:38:03 dist-test-slave-twwt krb5kdc[24289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459083, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:03.752629 24315 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:38:03.753192 24315 webserver.cc:460] Webserver started at http://127.2.103.126:40129/ using document root <none> and password file <none>
I20260811 14:38:03.753441 24315 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:03.753486 24315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:03.753583 24315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:38:03.754757 24315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "fe7128e743034d9cb7ca6c41df10a211"
format_stamp: "Formatted at 2026-08-11 14:38:03 on dist-test-slave-twwt"
server_key: "93159bb13e146d484620a09d7047649c"
server_key_iv: "d8ea73a242d0facb996f48ed4de5680f"
server_key_version: "encryptionkey@0"
I20260811 14:38:03.755091 24315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "fe7128e743034d9cb7ca6c41df10a211"
format_stamp: "Formatted at 2026-08-11 14:38:03 on dist-test-slave-twwt"
server_key: "93159bb13e146d484620a09d7047649c"
server_key_iv: "d8ea73a242d0facb996f48ed4de5680f"
server_key_version: "encryptionkey@0"
I20260811 14:38:03.756523 24315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:38:03.757292 24337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:03.757561 24315 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:03.757668 24315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "fe7128e743034d9cb7ca6c41df10a211"
format_stamp: "Formatted at 2026-08-11 14:38:03 on dist-test-slave-twwt"
server_key: "93159bb13e146d484620a09d7047649c"
server_key_iv: "d8ea73a242d0facb996f48ed4de5680f"
server_key_version: "encryptionkey@0"
I20260811 14:38:03.757746 24315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:03.770541 24315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:03.773150 24315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:03.773327 24315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:03.777817 24315 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:40961
I20260811 14:38:03.777900 24419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:40961 every 8 connection(s)
I20260811 14:38:03.778261 24315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:38:03.779299 24420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:03.781091 24420 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:38:03.785797 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 24315
I20260811 14:38:03.785979 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:38:03.786204 2461 external_mini_cluster.cc:1428] Setting key b93fb19b143e47626c0a8ab75a6d4eb6
Aug 11 14:38:03 dist-test-slave-twwt krb5kdc[24289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459083, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:38:03.791916 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:03.781922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:40961 (local address 127.0.0.1:60540)
0811 14:38:03.782107 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:03.782162 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:03.782801 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:03.783155 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:03.783159 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:03.783456 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:03.783628 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:03.783632 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:03.784653 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:03.784654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:03.785387 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:03.785389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:03.786019 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:03.786021 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:03.786107 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:03.786705 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:03.786717 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:03.788413 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:03.790298 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:03.790305 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:03.790307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:03.790511 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:03.790839 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:03.790839 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:03.790840 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:03.790934 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:03.791308 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:03.791313 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:03.791378 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 14:38:03.791491 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 14:38:03.791731 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":95,"thread_start_us":83,"threads_started":1}
I20260811 14:38:03.791914 24424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:03.782040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60540 (local address 127.2.103.126:40961)
0811 14:38:03.782242 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:03.782246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:03.782285 (+ 39us) server_negotiation.cc:407] Connection header received
0811 14:38:03.782984 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:03.782987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:03.783007 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:03.783105 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:03.783728 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:03.784560 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:03.785480 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:03.785861 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:03.786018 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:03.788521 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:03.788528 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:03.788529 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:03.788536 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:03.790188 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:03.790636 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:03.790639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:03.790640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:03.790774 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:03.791005 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:03.791006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:03.791006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:03.791232 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:03.791337 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:03.791518 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 14:38:03.791663 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":93,"threads_started":1}
I20260811 14:38:03.792765 24420 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:38:03.793160 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:03.793033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:43191 (local address 127.0.0.1:52520)
0811 14:38:03.793083 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:03.793128 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.125:43191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 14:38:03.793324 24341 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.103.125:43191: connect: Connection refused (error 111)
I20260811 14:38:03.793771 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:03.793675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:43191 (local address 127.0.0.1:52534)
0811 14:38:03.793724 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:03.793748 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.125:43191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 14:38:03.793891 24420 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.125:43191: Network error: Client connection negotiation failed: client connection to 127.2.103.125:43191: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.2.103.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.125.keytab.
Entry for principal kudu/127.2.103.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.125.keytab.
I20260811 14:38:03.814909 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.125:43191
--webserver_interface=127.2.103.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.125.keytab
--principal=kudu/127.2.103.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:43721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:40961,127.2.103.125:43191,127.2.103.124:33343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:38:03.843180 24420 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } attempt: 1
I20260811 14:38:03.843814 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:03.843653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:43191 (local address 127.0.0.1:52538)
0811 14:38:03.843725 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:03.843778 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.125:43191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 14:38:03.844246 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:03.844149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:43191 (local address 127.0.0.1:52548)
0811 14:38:03.844198 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:03.844227 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.125:43191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 14:38:03.844378 24420 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.125:43191: Network error: Client connection negotiation failed: client connection to 127.2.103.125:43191: connect: Connection refused (error 111)
W20260811 14:38:03.903405 24431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:03.903630 24431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:03.903654 24431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:03.904783 24431 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:38:03.904829 24431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:03.904843 24431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:03.904856 24431 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:38:03.904870 24431 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:38:03.906407 24431 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:43721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:40961,127.2.103.125:43191,127.2.103.124:33343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.125.keytab
--principal=kudu/127.2.103.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.125:43191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.103.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:03.906701 24431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:03.906927 24431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:03.912298 24439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:03.912433 24431 server_base.cc:1034] running on GCE node
W20260811 14:38:03.912287 24447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:03.912348 24441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:03.912632 24420 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } attempt: 2
I20260811 14:38:03.912878 24431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:03.913126 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:03.912971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:43191 (local address 127.0.0.1:52564)
0811 14:38:03.913044 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:03.913085 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.125:43191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 14:38:03.913278 24431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:03.913563 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:03.913449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:43191 (local address 127.0.0.1:52568)
0811 14:38:03.913504 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:03.913537 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.125:43191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260811 14:38:03.913707 24420 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.125:43191: Network error: Client connection negotiation failed: client connection to 127.2.103.125:43191: connect: Connection refused (error 111)
I20260811 14:38:03.914449 24431 hybrid_clock.cc:648] HybridClock initialized: now 1786459083914420 us; error 41 us; skew 500 ppm
Aug 11 14:38:03 dist-test-slave-twwt krb5kdc[24289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459083, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:03.917214 24431 init.cc:377] Logged in from keytab as kudu/127.2.103.125@KRBTEST.COM (short username kudu)
I20260811 14:38:03.917692 24431 webserver.cc:460] Webserver started at http://127.2.103.125:40047/ using document root <none> and password file <none>
I20260811 14:38:03.917976 24431 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:03.918026 24431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:03.918211 24431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:38:03.919445 24431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-1/data/instance:
uuid: "ea662feef408411b93767ee612278b2d"
format_stamp: "Formatted at 2026-08-11 14:38:03 on dist-test-slave-twwt"
server_key: "b6a73bf8a54f4e82deadc4e9e30a4ae0"
server_key_iv: "16c088266dc8b84e8693435b4408e6e4"
server_key_version: "encryptionkey@0"
I20260811 14:38:03.919888 24431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-1/wal/instance:
uuid: "ea662feef408411b93767ee612278b2d"
format_stamp: "Formatted at 2026-08-11 14:38:03 on dist-test-slave-twwt"
server_key: "b6a73bf8a54f4e82deadc4e9e30a4ae0"
server_key_iv: "16c088266dc8b84e8693435b4408e6e4"
server_key_version: "encryptionkey@0"
I20260811 14:38:03.921324 24431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:38:03.922108 24459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:03.922297 24431 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:38:03.922413 24431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-1/wal
uuid: "ea662feef408411b93767ee612278b2d"
format_stamp: "Formatted at 2026-08-11 14:38:03 on dist-test-slave-twwt"
server_key: "b6a73bf8a54f4e82deadc4e9e30a4ae0"
server_key_iv: "16c088266dc8b84e8693435b4408e6e4"
server_key_version: "encryptionkey@0"
I20260811 14:38:03.922492 24431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:03.945570 24431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:03.948016 24431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:03.948215 24431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:03.953253 24431 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.125:43191
I20260811 14:38:03.953327 24544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.125:43191 every 8 connection(s)
I20260811 14:38:03.954012 24431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-1/data/info.pb
I20260811 14:38:03.954888 24545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:03.957331 24545 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:38:03.961404 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 24431
I20260811 14:38:03.961535 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-1/wal/instance
I20260811 14:38:03.961768 2461 external_mini_cluster.cc:1428] Setting key 9c8d11d28f6564a8f487eec3c92060ca
Aug 11 14:38:03 dist-test-slave-twwt krb5kdc[24289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459083, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.125@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:38:03.967948 24424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:03.958565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60552 (local address 127.2.103.126:40961)
0811 14:38:03.958649 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:03.958651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:03.958662 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:38:03.959337 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:03.959339 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:03.959382 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:03.959438 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:03.960385 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:03.960982 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:03.962092 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:03.962321 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:03.962403 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:03.964894 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:03.964896 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:03.964896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:03.964903 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:03.966259 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:03.966829 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:03.966830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:03.966831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:03.966889 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:03.967185 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:03.967185 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:03.967185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:03.967286 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:03.967333 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:03.967785 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 14:38:03.967874 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 14:38:03.968169 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:03.958214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:40961 (local address 127.0.0.1:60552)
0811 14:38:03.958443 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:03.958495 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:03.959180 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:03.959509 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:03.959514 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:03.959926 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:03.960237 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:03.960243 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:03.961100 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:03.961102 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:03.962002 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:03.962004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:03.962409 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:03.962410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:03.962549 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:03.963182 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:03.963192 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:03.964795 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:03.966350 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:03.966358 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:03.966361 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:03.966736 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:03.966965 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:03.966966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:03.966967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:03.967119 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:03.967350 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:03.967358 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:03.967545 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 14:38:03.967669 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 14:38:03.967874 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":114,"threads_started":1}
I20260811 14:38:03.968804 24545 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.103.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.124@KRBTEST.COM" created.
Aug 11 14:38:03 dist-test-slave-twwt krb5kdc[24289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459083, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.125@KRBTEST.COM for kudu/127.2.103.125@KRBTEST.COM
I20260811 14:38:03.978365 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:03.969136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:43191 (local address 127.0.0.1:52578)
0811 14:38:03.969217 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:03.969231 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:03.969282 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:03.969584 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:03.969585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:03.969737 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:03.969791 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:03.969793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:03.970722 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:03.970724 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:03.971294 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:03.971297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:03.971848 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:03.971849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:03.971921 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:03.972337 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:03.972342 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:03.975039 (+ 2697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:03.977175 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:03.977176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:03.977177 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:03.977366 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:03.977611 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:03.977611 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:03.977612 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:03.977670 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:03.978080 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:03.978081 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:03.978127 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 14:38:03.978196 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 14:38:03.978284 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 14:38:03.978672 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:03.969266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52578 (local address 127.2.103.125:43191)
0811 14:38:03.969444 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:03.969448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:03.969465 (+ 17us) server_negotiation.cc:407] Connection header received
0811 14:38:03.969483 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:03.969484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:03.969495 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:03.969584 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:03.969878 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:03.970565 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:03.971375 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:03.971770 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:03.971852 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:03.975136 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:03.975147 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:03.975147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:03.975154 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:03.977045 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:03.977456 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:03.977459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:03.977460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:03.977531 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:03.977746 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:03.977747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:03.977747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:03.977972 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:03.978063 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:03.978259 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 14:38:03.978358 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":96,"threads_started":1}
I20260811 14:38:03.979473 24545 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:38:03.979959 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:03.979788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33343 (local address 127.0.0.1:44072)
0811 14:38:03.979853 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:03.979923 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 14:38:03.980093 24460 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.103.124:33343: connect: Connection refused (error 111)
I20260811 14:38:03.980650 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:03.980510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33343 (local address 127.0.0.1:44082)
0811 14:38:03.980596 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:03.980628 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 14:38:03.980844 24545 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.124:33343: Network error: Client connection negotiation failed: client connection to 127.2.103.124:33343: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.124.keytab.
Entry for principal kudu/127.2.103.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.124.keytab.
I20260811 14:38:03.994382 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.124:33343
--webserver_interface=127.2.103.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.124.keytab
--principal=kudu/127.2.103.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:43721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:40961,127.2.103.125:43191,127.2.103.124:33343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:38:04.004879 24420 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } attempt: 3
Aug 11 14:38:04 dist-test-slave-twwt krb5kdc[24289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459083, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for kudu/127.2.103.125@KRBTEST.COM
I20260811 14:38:04.012516 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.005310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:43191 (local address 127.0.0.1:52594)
0811 14:38:04.005385 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:04.005404 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:04.005476 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:04.005666 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:04.005668 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:04.005841 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:04.005919 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:04.005922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.006639 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.006640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:04.007149 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:04.007151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.007546 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.007547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:04.007633 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:04.008146 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:04.008150 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:04.009597 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:04.011545 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:04.011546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:04.011547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:04.011721 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:04.011928 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:04.011929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:04.011929 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:04.011986 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:04.012252 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:04.012253 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:04.012298 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 14:38:04.012355 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 14:38:04.012442 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 14:38:04.012590 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.005398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52594 (local address 127.2.103.125:43191)
0811 14:38:04.005470 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:04.005473 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:04.005486 (+ 13us) server_negotiation.cc:407] Connection header received
0811 14:38:04.005571 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:04.005576 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:04.005586 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:04.005645 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:04.006023 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.006523 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.007243 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.007465 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.007548 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:04.009694 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:04.009696 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:04.009696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:04.009705 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:04.011445 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.011791 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.011792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.011792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.011856 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.012045 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.012046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.012046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.012184 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:04.012251 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:04.012403 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 14:38:04.012492 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 14:38:04.012943 24420 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:38:04.013304 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.013202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33343 (local address 127.0.0.1:44088)
0811 14:38:04.013249 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:04.013282 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 14:38:04.013659 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.013579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33343 (local address 127.0.0.1:44094)
0811 14:38:04.013620 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:04.013644 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 14:38:04.013770 24420 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.124:33343: Network error: Client connection negotiation failed: client connection to 127.2.103.124:33343: connect: Connection refused (error 111)
I20260811 14:38:04.030094 24545 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } attempt: 1
I20260811 14:38:04.030689 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.030500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33343 (local address 127.0.0.1:44102)
0811 14:38:04.030576 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:04.030658 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 14:38:04.031067 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.030980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33343 (local address 127.0.0.1:44106)
0811 14:38:04.031022 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:04.031050 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260811 14:38:04.031172 24545 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.124:33343: Network error: Client connection negotiation failed: client connection to 127.2.103.124:33343: connect: Connection refused (error 111)
I20260811 14:38:04.044912 24420 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } attempt: 1
I20260811 14:38:04.045377 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.045239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33343 (local address 127.0.0.1:44108)
0811 14:38:04.045303 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:04.045337 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 14:38:04.045698 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.045607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33343 (local address 127.0.0.1:44114)
0811 14:38:04.045651 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:04.045675 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260811 14:38:04.045796 24420 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.124:33343: Network error: Client connection negotiation failed: client connection to 127.2.103.124:33343: connect: Connection refused (error 111)
W20260811 14:38:04.085649 24562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:04.085935 24562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:04.085976 24562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:04.087249 24562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:38:04.087328 24562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:04.087353 24562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:04.087374 24562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:38:04.087404 24562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:38:04.089007 24562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:43721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:40961,127.2.103.125:43191,127.2.103.124:33343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.124.keytab
--principal=kudu/127.2.103.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.124:33343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.103.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:04.089313 24562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:04.089560 24562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:04.094895 24571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:04.094913 24574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:04.094971 24570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:04.095139 24562 server_base.cc:1034] running on GCE node
I20260811 14:38:04.095408 24562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:04.095722 24562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:04.096887 24562 hybrid_clock.cc:648] HybridClock initialized: now 1786459084096863 us; error 36 us; skew 500 ppm
Aug 11 14:38:04 dist-test-slave-twwt krb5kdc[24289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459084, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:04.099336 24545 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } attempt: 2
I20260811 14:38:04.099723 24562 init.cc:377] Logged in from keytab as kudu/127.2.103.124@KRBTEST.COM (short username kudu)
I20260811 14:38:04.099787 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.099638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33343 (local address 127.0.0.1:44118)
0811 14:38:04.099703 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:04.099740 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 14:38:04.100181 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.100075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33343 (local address 127.0.0.1:44128)
0811 14:38:04.100125 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:04.100151 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 14:38:04.100238 24562 webserver.cc:460] Webserver started at http://127.2.103.124:44525/ using document root <none> and password file <none>
W20260811 14:38:04.100317 24545 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.124:33343: Network error: Client connection negotiation failed: client connection to 127.2.103.124:33343: connect: Connection refused (error 111)
I20260811 14:38:04.100541 24562 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:04.100591 24562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:04.100744 24562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:38:04.101861 24562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-2/data/instance:
uuid: "bc5906ba952e4842a0b7995127ac58b6"
format_stamp: "Formatted at 2026-08-11 14:38:04 on dist-test-slave-twwt"
server_key: "35b9131c5b4366adf95864d4f11f47da"
server_key_iv: "72c3a6a1e32a7a00d88e268ce2600503"
server_key_version: "encryptionkey@0"
I20260811 14:38:04.102252 24562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-2/wal/instance:
uuid: "bc5906ba952e4842a0b7995127ac58b6"
format_stamp: "Formatted at 2026-08-11 14:38:04 on dist-test-slave-twwt"
server_key: "35b9131c5b4366adf95864d4f11f47da"
server_key_iv: "72c3a6a1e32a7a00d88e268ce2600503"
server_key_version: "encryptionkey@0"
I20260811 14:38:04.103788 24562 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260811 14:38:04.104666 24587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:04.104897 24562 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:04.104997 24562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-2/wal
uuid: "bc5906ba952e4842a0b7995127ac58b6"
format_stamp: "Formatted at 2026-08-11 14:38:04 on dist-test-slave-twwt"
server_key: "35b9131c5b4366adf95864d4f11f47da"
server_key_iv: "72c3a6a1e32a7a00d88e268ce2600503"
server_key_version: "encryptionkey@0"
I20260811 14:38:04.105084 24562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:04.113225 24562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:04.115676 24562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:04.115818 24562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:04.120626 24562 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.124:33343
I20260811 14:38:04.120702 24663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.124:33343 every 8 connection(s)
I20260811 14:38:04.120966 24420 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } attempt: 2
I20260811 14:38:04.121138 24562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-2/data/info.pb
I20260811 14:38:04.122153 24664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:04.124003 24664 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 14:38:04 dist-test-slave-twwt krb5kdc[24289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459083, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for kudu/127.2.103.124@KRBTEST.COM
Aug 11 14:38:04 dist-test-slave-twwt krb5kdc[24289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459084, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.124@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:38:04.130575 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 24562
I20260811 14:38:04.130739 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458919184199-2461-0/minicluster-data/master-2/wal/instance
I20260811 14:38:04.130959 2461 external_mini_cluster.cc:1428] Setting key 1f93393671694c87d3724efedb356df0
I20260811 14:38:04.131691 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.121361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33343 (local address 127.0.0.1:44134)
0811 14:38:04.121441 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:04.121456 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:04.121526 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:04.122650 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:04.122652 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:04.122845 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:04.122905 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:04.122908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.124077 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.124078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:04.124944 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:04.124948 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.125714 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.125715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:04.125787 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:04.126277 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:04.126283 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:04.127948 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:04.130073 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:04.130074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:04.130074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:04.130370 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:04.130743 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:04.130745 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:04.130745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:04.130819 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:04.131367 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:04.131368 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:04.131420 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 14:38:04.131473 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 14:38:04.131567 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 14:38:04.132375 24667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.121526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44134 (local address 127.2.103.124:33343)
0811 14:38:04.121770 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:04.121781 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:04.121818 (+ 37us) server_negotiation.cc:407] Connection header received
0811 14:38:04.122407 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:04.122416 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:04.122472 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:04.122662 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:04.123041 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.123967 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.125040 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.125633 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.125718 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:04.128082 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:04.128092 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:04.128092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:04.128106 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:04.129997 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.130465 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.130469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.130469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.130649 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.130904 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.130905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.130905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.131266 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:04.131851 (+ 585us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:04.131938 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 14:38:04.132102 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":139,"threads_started":1}
I20260811 14:38:04.133312 24424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.124869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60568 (local address 127.2.103.126:40961)
0811 14:38:04.124926 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:04.124928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:04.125160 (+ 232us) server_negotiation.cc:407] Connection header received
0811 14:38:04.125203 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:04.125203 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:04.125215 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:04.125271 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:04.125790 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.126281 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.127306 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.127638 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.127708 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:04.129981 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:04.129983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:04.129983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:04.129990 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:04.131900 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.132337 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.132338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.132338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.132396 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.132611 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.132612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.132612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.132838 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:04.132889 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:04.133126 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 14:38:04.133207 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 14:38:04.133611 24671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.124847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:40961 (local address 127.0.0.1:60568)
0811 14:38:04.125050 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:04.125095 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:04.125152 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:04.125285 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:04.125287 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:04.125596 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:04.125699 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:04.125702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.126382 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.126383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:04.127206 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:04.127209 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.127723 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.127724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:04.127806 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:04.128422 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:04.128426 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:04.129889 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:04.131996 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:04.132002 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:04.132004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:04.132246 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:04.132477 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:04.132478 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:04.132478 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:04.132540 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:04.132940 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:04.132946 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:04.132997 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 14:38:04.133395 (+ 398us) client_negotiation.cc:240] Negotiation successful
0811 14:38:04.133512 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":108,"threads_started":1}
I20260811 14:38:04.134560 24420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211: Bootstrap starting.
I20260811 14:38:04.135293 24420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:04.135548 24420 log.cc:824] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:04.136016 24664 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:38:04.136471 24420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211: No bootstrap required, opened a new log
Aug 11 14:38:04 dist-test-slave-twwt krb5kdc[24289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.124@KRBTEST.COM
I20260811 14:38:04.138546 24420 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe7128e743034d9cb7ca6c41df10a211" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } } peers { permanent_uuid: "ea662feef408411b93767ee612278b2d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } } peers { permanent_uuid: "bc5906ba952e4842a0b7995127ac58b6" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } }
I20260811 14:38:04.138739 24420 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:04.138798 24420 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe7128e743034d9cb7ca6c41df10a211, State: Initialized, Role: FOLLOWER
I20260811 14:38:04.138958 24420 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe7128e743034d9cb7ca6c41df10a211" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } } peers { permanent_uuid: "ea662feef408411b93767ee612278b2d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } } peers { permanent_uuid: "bc5906ba952e4842a0b7995127ac58b6" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } }
I20260811 14:38:04.139318 24675 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe7128e743034d9cb7ca6c41df10a211" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } } peers { permanent_uuid: "ea662feef408411b93767ee612278b2d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } } peers { permanent_uuid: "bc5906ba952e4842a0b7995127ac58b6" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } } }
I20260811 14:38:04.139971 24675 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:38:04.139374 24420 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211 [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 14:38:04 dist-test-slave-twwt krb5kdc[24289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459084, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.124@KRBTEST.COM for kudu/127.2.103.125@KRBTEST.COM
I20260811 14:38:04.142172 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.131554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44138 (local address 127.2.103.124:33343)
0811 14:38:04.131704 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:04.131706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:04.131757 (+ 51us) server_negotiation.cc:407] Connection header received
0811 14:38:04.131888 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:04.131889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:04.131900 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:04.132670 (+ 770us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:04.132721 (+ 51us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.133267 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.134335 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.134526 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.134600 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:04.137955 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:04.137956 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:04.137957 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:04.137963 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:04.140776 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.141239 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.141240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.141241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.141308 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.141528 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.141529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.141529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.141685 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:04.141856 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:04.141954 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 14:38:04.142046 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":102,"threads_started":1}
W20260811 14:38:04.142759 24690 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:38:04.142815 24690 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:38:04.144357 24671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.136296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:43191 (local address 127.0.0.1:52600)
0811 14:38:04.136336 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:04.136346 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:04.136381 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:04.136560 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:04.136561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:04.136717 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:04.136788 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:04.136790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.137530 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.137531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:04.138330 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:04.138333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.138829 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.138830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:04.138938 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:04.139379 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:04.139384 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:04.140881 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:04.143276 (+ 2395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:04.143277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:04.143278 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:04.143487 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:04.143698 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:04.143698 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:04.143699 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:04.143754 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:04.144043 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:04.144044 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:04.144091 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 14:38:04.144153 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 14:38:04.144267 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260811 14:38:04.144666 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.136368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52600 (local address 127.2.103.125:43191)
0811 14:38:04.136416 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:04.136418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:04.136427 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:38:04.136462 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:04.136463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:04.136474 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:04.136538 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:04.136932 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.137433 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.138455 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.138715 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.138801 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:04.141006 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:04.141007 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:04.141008 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:04.141016 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:04.143165 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.143571 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.143573 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.143573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.143633 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.143820 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.143821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.143821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.143955 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:04.144014 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:04.144489 (+ 475us) server_negotiation.cc:299] Negotiation successful
0811 14:38:04.144578 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:38:04.143913 24315 master_runner.cc:186] Error getting master registration for 127.2.103.125:43191: OK, instance_id { permanent_uuid: "ea662feef408411b93767ee612278b2d" instance_seqno: 1786459083951651 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 14:38:04.146867 24315 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 14:38:04.147099 24664 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 14:38:04 dist-test-slave-twwt krb5kdc[24289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
Aug 11 14:38:04 dist-test-slave-twwt krb5kdc[24289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459084, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.124@KRBTEST.COM for kudu/127.2.103.124@KRBTEST.COM
I20260811 14:38:04.152354 24424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.143523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60570 (local address 127.2.103.126:40961)
0811 14:38:04.143564 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:04.143566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:04.143574 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:38:04.143594 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:04.143595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:04.143606 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:04.143650 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:04.144349 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.145149 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.146181 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.146456 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.146534 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:04.149287 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:04.149289 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:04.149290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:04.149299 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:04.150967 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.151424 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.151426 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.151426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.151497 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.151740 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.151742 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.151742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.151891 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:04.152085 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:04.152155 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 14:38:04.152254 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 14:38:04.155983 24671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.147404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33343 (local address 127.0.0.1:44148)
0811 14:38:04.147487 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:04.147497 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:04.147535 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:04.148207 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:04.148208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:04.148357 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:04.148388 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:04.148389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.149137 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.149138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:04.149854 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:04.149858 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.150670 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.150672 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:04.150738 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:04.151179 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:04.151184 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:04.152660 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:04.154735 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:04.154736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:04.154737 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:04.154962 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:04.155199 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:04.155199 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:04.155200 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:04.155267 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:04.155629 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:04.155630 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:04.155677 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 14:38:04.155756 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 14:38:04.155865 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 14:38:04.156066 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.147522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44148 (local address 127.2.103.124:33343)
0811 14:38:04.148087 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:04.148089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:04.148096 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:38:04.148115 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:04.148116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:04.148125 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:04.148205 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:04.148472 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.149055 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.149950 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.150553 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.150659 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:04.152778 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:04.152779 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:04.152779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:04.152787 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:04.154568 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.155045 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.155046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.155047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.155112 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.155336 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.155337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.155338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.155544 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:04.155619 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:04.155803 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 14:38:04.155934 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":542}
I20260811 14:38:04.158087 24664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6: Bootstrap starting.
Aug 11 14:38:04 dist-test-slave-twwt krb5kdc[24289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.125@KRBTEST.COM
I20260811 14:38:04.158955 24664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:04.159274 24664 log.cc:824] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:04.160318 24664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6: No bootstrap required, opened a new log
I20260811 14:38:04.162169 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.153647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52602 (local address 127.2.103.125:43191)
0811 14:38:04.153703 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:04.153705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:04.153739 (+ 34us) server_negotiation.cc:407] Connection header received
0811 14:38:04.153837 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:04.153838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:04.153850 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:04.153896 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:04.154442 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.155250 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.156158 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.156458 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.156522 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:04.159103 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:04.159104 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:04.159105 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:04.159113 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:04.160809 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.161237 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.161238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.161239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.161319 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.161556 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.161557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.161557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.161727 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:04.161931 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:04.161970 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 14:38:04.162051 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 14:38:04.162370 24664 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe7128e743034d9cb7ca6c41df10a211" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } } peers { permanent_uuid: "ea662feef408411b93767ee612278b2d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } } peers { permanent_uuid: "bc5906ba952e4842a0b7995127ac58b6" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } }
I20260811 14:38:04.162536 24664 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:04.162571 24664 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc5906ba952e4842a0b7995127ac58b6, State: Initialized, Role: FOLLOWER
I20260811 14:38:04.162724 24664 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe7128e743034d9cb7ca6c41df10a211" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } } peers { permanent_uuid: "ea662feef408411b93767ee612278b2d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } } peers { permanent_uuid: "bc5906ba952e4842a0b7995127ac58b6" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } }
I20260811 14:38:04.163055 24664 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:38:04.163018 24693 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe7128e743034d9cb7ca6c41df10a211" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } } peers { permanent_uuid: "ea662feef408411b93767ee612278b2d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } } peers { permanent_uuid: "bc5906ba952e4842a0b7995127ac58b6" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } } }
I20260811 14:38:04.163244 24693 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:38:04.163548 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260811 14:38:04.165151 24718 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:38:04.165227 24718 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:38:04.166401 24562 master_runner.cc:186] Error getting master registration for 127.2.103.125:43191: OK, instance_id { permanent_uuid: "ea662feef408411b93767ee612278b2d" instance_seqno: 1786459083951651 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 14:38:04.168676 24562 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 14:38:04.178519 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.167530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44160 (local address 127.2.103.124:33343)
0811 14:38:04.167566 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:04.167568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:04.170949 (+ 3381us) server_negotiation.cc:407] Connection header received
0811 14:38:04.171040 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:04.171040 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:04.171050 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:04.171079 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:04.171971 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.172699 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.173722 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.174022 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.174064 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:04.175686 (+ 1622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:04.175687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:04.175688 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:04.175695 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:04.177289 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.177674 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.177675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.177676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.177729 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.177922 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.177923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.177923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.178042 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:04.178112 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:04.178331 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 14:38:04.178419 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 14:38:04.181703 24424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.166910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60584 (local address 127.2.103.126:40961)
0811 14:38:04.166982 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:04.166984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:04.167937 (+ 953us) server_negotiation.cc:407] Connection header received
0811 14:38:04.168063 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:04.168065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:04.168076 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:04.168109 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:04.169043 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.169807 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.173521 (+ 3714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.173857 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.173918 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:04.178095 (+ 4177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:04.178097 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:04.178097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:04.178106 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:04.180366 (+ 2260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.180822 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.180823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.180823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.180884 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.181103 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.181104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.181105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.181236 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:04.181305 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:04.181511 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 14:38:04.181597 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:38:04.183494 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.167182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52606 (local address 127.2.103.125:43191)
0811 14:38:04.167220 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:04.167222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:04.170808 (+ 3586us) server_negotiation.cc:407] Connection header received
0811 14:38:04.170889 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:04.170891 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:04.170903 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:04.170970 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:04.171741 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.172471 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.174759 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.175077 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.175122 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:04.180511 (+ 5389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:04.180513 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:04.180514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:04.180523 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:04.182201 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.182599 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.182600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.182600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.182686 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.182890 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.182891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.182892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.183020 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:04.183100 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:04.183310 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 14:38:04.183396 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 14:38:04.191461 24545 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } attempt: 3
Aug 11 14:38:04 dist-test-slave-twwt krb5kdc[24289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459083, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.125@KRBTEST.COM for kudu/127.2.103.124@KRBTEST.COM
I20260811 14:38:04.200232 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.191792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:33343 (local address 127.0.0.1:44176)
0811 14:38:04.191852 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:04.191863 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:04.191902 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:04.192062 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:04.192064 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:04.192265 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:04.192321 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:04.192323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.193294 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.193295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:04.194120 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:04.194122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.194652 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.194653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:04.194716 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:04.195255 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:04.195261 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:04.196943 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:04.199025 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:04.199026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:04.199027 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:04.199268 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:04.199505 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:04.199506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:04.199506 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:04.199580 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:04.199872 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:04.199874 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:04.199928 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 14:38:04.200010 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 14:38:04.200120 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 14:38:04.200235 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:04.191871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44176 (local address 127.2.103.124:33343)
0811 14:38:04.191930 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:04.191933 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:04.191942 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:38:04.191968 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:04.191968 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:04.191979 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:04.192044 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:04.192411 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.193177 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.194211 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:04.194532 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:04.194643 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:04.197054 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:04.197055 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:04.197055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:04.197064 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:04.198931 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.199352 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.199353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.199354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.199420 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:04.199652 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:04.199653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:04.199654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:04.199798 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:04.199868 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:04.200051 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 14:38:04.200139 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:38:04.202400 24545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d: Bootstrap starting.
I20260811 14:38:04.203385 24545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:04.203723 24545 log.cc:824] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:04.205066 24545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d: No bootstrap required, opened a new log
I20260811 14:38:04.207242 24545 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe7128e743034d9cb7ca6c41df10a211" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } } peers { permanent_uuid: "ea662feef408411b93767ee612278b2d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } } peers { permanent_uuid: "bc5906ba952e4842a0b7995127ac58b6" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } }
I20260811 14:38:04.207402 24545 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:04.207438 24545 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea662feef408411b93767ee612278b2d, State: Initialized, Role: FOLLOWER
I20260811 14:38:04.207602 24545 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe7128e743034d9cb7ca6c41df10a211" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } } peers { permanent_uuid: "ea662feef408411b93767ee612278b2d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } } peers { permanent_uuid: "bc5906ba952e4842a0b7995127ac58b6" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } }
I20260811 14:38:04.207983 24545 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:38:04.207930 24735 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe7128e743034d9cb7ca6c41df10a211" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } } peers { permanent_uuid: "ea662feef408411b93767ee612278b2d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } } peers { permanent_uuid: "bc5906ba952e4842a0b7995127ac58b6" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } } }
I20260811 14:38:04.208158 24735 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d [sys.catalog]: This master's current role is: FOLLOWER
W20260811 14:38:04.210049 24750 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:38:04.210172 24750 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:38:04.212152 24431 master_runner.cc:263] No leader master found from master ea662feef408411b93767ee612278b2d
I20260811 14:38:04.212206 24431 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 14:38:04.249961 24693 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:38:04.250113 24693 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe7128e743034d9cb7ca6c41df10a211" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } } peers { permanent_uuid: "ea662feef408411b93767ee612278b2d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } } peers { permanent_uuid: "bc5906ba952e4842a0b7995127ac58b6" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } }
I20260811 14:38:04.250412 24693 leader_election.cc:288] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe7128e743034d9cb7ca6c41df10a211 (127.2.103.126:40961), ea662feef408411b93767ee612278b2d (127.2.103.125:43191)
I20260811 14:38:04.251019 24385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc5906ba952e4842a0b7995127ac58b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe7128e743034d9cb7ca6c41df10a211" is_pre_election: true
I20260811 14:38:04.251194 24385 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc5906ba952e4842a0b7995127ac58b6 in term 0.
I20260811 14:38:04.251468 24588 leader_election.cc:302] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc5906ba952e4842a0b7995127ac58b6, fe7128e743034d9cb7ca6c41df10a211; no voters:
I20260811 14:38:04.252079 24508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc5906ba952e4842a0b7995127ac58b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea662feef408411b93767ee612278b2d" is_pre_election: true
I20260811 14:38:04.252264 24508 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc5906ba952e4842a0b7995127ac58b6 in term 0.
I20260811 14:38:04.252614 24693 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:38:04.252672 24693 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:38:04.252698 24693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:38:04.253657 24693 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe7128e743034d9cb7ca6c41df10a211" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } } peers { permanent_uuid: "ea662feef408411b93767ee612278b2d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } } peers { permanent_uuid: "bc5906ba952e4842a0b7995127ac58b6" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } }
I20260811 14:38:04.253795 24693 leader_election.cc:288] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [CANDIDATE]: Term 1 election: Requested vote from peers fe7128e743034d9cb7ca6c41df10a211 (127.2.103.126:40961), ea662feef408411b93767ee612278b2d (127.2.103.125:43191)
I20260811 14:38:04.254143 24385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc5906ba952e4842a0b7995127ac58b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe7128e743034d9cb7ca6c41df10a211"
I20260811 14:38:04.254230 24385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:38:04.255223 24385 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc5906ba952e4842a0b7995127ac58b6 in term 1.
I20260811 14:38:04.255435 24588 leader_election.cc:302] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc5906ba952e4842a0b7995127ac58b6, fe7128e743034d9cb7ca6c41df10a211; no voters:
I20260811 14:38:04.255678 24508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc5906ba952e4842a0b7995127ac58b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea662feef408411b93767ee612278b2d"
I20260811 14:38:04.255764 24508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:38:04.258368 24693 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:38:04.258550 24693 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [term 1 LEADER]: Becoming Leader. State: Replica: bc5906ba952e4842a0b7995127ac58b6, State: Running, Role: LEADER
I20260811 14:38:04.258667 24693 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe7128e743034d9cb7ca6c41df10a211" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } } peers { permanent_uuid: "ea662feef408411b93767ee612278b2d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } } peers { permanent_uuid: "bc5906ba952e4842a0b7995127ac58b6" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } }
I20260811 14:38:04.259099 24693 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc5906ba952e4842a0b7995127ac58b6. Latest consensus state: current_term: 1 leader_uuid: "bc5906ba952e4842a0b7995127ac58b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe7128e743034d9cb7ca6c41df10a211" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } } peers { permanent_uuid: "ea662feef408411b93767ee612278b2d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } } peers { permanent_uuid: "bc5906ba952e4842a0b7995127ac58b6" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } } }
I20260811 14:38:04.259168 24693 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [sys.catalog]: This master's current role is: LEADER
I20260811 14:38:04.259711 24508 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc5906ba952e4842a0b7995127ac58b6 in term 1.
I20260811 14:38:04.260840 24755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:38:04.261068 24755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:38:04.263165 24508 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d [term 1 FOLLOWER]: Refusing update from remote peer bc5906ba952e4842a0b7995127ac58b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:38:04.263620 24693 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea662feef408411b93767ee612278b2d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:38:04.265233 24735 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc5906ba952e4842a0b7995127ac58b6. Latest consensus state: current_term: 1 leader_uuid: "bc5906ba952e4842a0b7995127ac58b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe7128e743034d9cb7ca6c41df10a211" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } } peers { permanent_uuid: "ea662feef408411b93767ee612278b2d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } } peers { permanent_uuid: "bc5906ba952e4842a0b7995127ac58b6" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } } }
I20260811 14:38:04.265340 24735 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:38:04.265885 24385 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211 [term 1 FOLLOWER]: Refusing update from remote peer bc5906ba952e4842a0b7995127ac58b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:38:04.266180 24693 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe7128e743034d9cb7ca6c41df10a211" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:38:04.267508 24675 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc5906ba952e4842a0b7995127ac58b6. Latest consensus state: current_term: 1 leader_uuid: "bc5906ba952e4842a0b7995127ac58b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe7128e743034d9cb7ca6c41df10a211" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } } peers { permanent_uuid: "ea662feef408411b93767ee612278b2d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } } peers { permanent_uuid: "bc5906ba952e4842a0b7995127ac58b6" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } } }
I20260811 14:38:04.267601 24675 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:38:04.269520 24753 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc5906ba952e4842a0b7995127ac58b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe7128e743034d9cb7ca6c41df10a211" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } } peers { permanent_uuid: "ea662feef408411b93767ee612278b2d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } } peers { permanent_uuid: "bc5906ba952e4842a0b7995127ac58b6" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } } }
I20260811 14:38:04.269629 24753 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [sys.catalog]: This master's current role is: LEADER
I20260811 14:38:04.270258 24735 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc5906ba952e4842a0b7995127ac58b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe7128e743034d9cb7ca6c41df10a211" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } } peers { permanent_uuid: "ea662feef408411b93767ee612278b2d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } } peers { permanent_uuid: "bc5906ba952e4842a0b7995127ac58b6" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } } }
I20260811 14:38:04.270375 24735 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:38:04.270653 24760 mvcc.cc:205] Tried to move back new op lower bound from 7317336409139302400 to 7317336409124130816. Current Snapshot: MvccSnapshot[applied={T|T < 7317336409139302400}]
I20260811 14:38:04.270802 24761 mvcc.cc:205] Tried to move back new op lower bound from 7317336409139302400 to 7317336409124130816. Current Snapshot: MvccSnapshot[applied={T|T < 7317336409139302400}]
I20260811 14:38:04.271277 24693 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc5906ba952e4842a0b7995127ac58b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe7128e743034d9cb7ca6c41df10a211" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } } peers { permanent_uuid: "ea662feef408411b93767ee612278b2d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } } peers { permanent_uuid: "bc5906ba952e4842a0b7995127ac58b6" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } } }
I20260811 14:38:04.271356 24693 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6 [sys.catalog]: This master's current role is: LEADER
I20260811 14:38:04.271755 24675 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc5906ba952e4842a0b7995127ac58b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe7128e743034d9cb7ca6c41df10a211" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40961 } } peers { permanent_uuid: "ea662feef408411b93767ee612278b2d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 43191 } } peers { permanent_uuid: "bc5906ba952e4842a0b7995127ac58b6" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 33343 } } }
I20260811 14:38:04.271857 24675 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:38:04.272644 24766 mvcc.cc:205] Tried to move back new op lower bound from 7317336409139302400 to 7317336409124130816. Current Snapshot: MvccSnapshot[applied={T|T < 7317336409139302400}]
I20260811 14:38:04.274835 24755 catalog_manager.cc:1347] Generated new cluster ID: 7c8104cc40704b1bbbebd0f2f2568908
I20260811 14:38:04.274886 24755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:38:04.297416 24755 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:38:04.298265 24755 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:38:04.311822 24755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc5906ba952e4842a0b7995127ac58b6: Generated new TSK 0
I20260811 14:38:04.312147 24755 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:38:05.143265 24690 catalog_manager.cc:1259] Loaded cluster ID: 7c8104cc40704b1bbbebd0f2f2568908
I20260811 14:38:05.143344 24690 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211: loading cluster ID for follower catalog manager: success
I20260811 14:38:05.144214 24690 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211: acquiring CA information for follower catalog manager: success
I20260811 14:38:05.144584 24690 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fe7128e743034d9cb7ca6c41df10a211: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 14:38:05.210748 24750 catalog_manager.cc:1259] Loaded cluster ID: 7c8104cc40704b1bbbebd0f2f2568908
I20260811 14:38:05.210847 24750 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d: loading cluster ID for follower catalog manager: success
I20260811 14:38:05.211735 24750 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d: acquiring CA information for follower catalog manager: success
I20260811 14:38:05.212075 24750 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ea662feef408411b93767ee612278b2d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 14:38:05.372541 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 24315
I20260811 14:38:05.390339 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 24431
I20260811 14:38:05.406379 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 24562
2026-08-11T14:38:05Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3167 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:38:05 dist-test-slave-twwt krb5kdc[24784](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:38:05 dist-test-slave-twwt krb5kdc[24784](info): set up 2 sockets
Aug 11 14:38:05 dist-test-slave-twwt krb5kdc[24784](info): commencing operation
krb5kdc: starting...
W20260811 14:38:06.685045 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:38:06 dist-test-slave-twwt krb5kdc[24784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:38:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:38:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:38:06.818478 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:37255
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:33733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:37255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:38:06.904836 24803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:06.905053 24803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:06.905076 24803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:06.906159 24803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:38:06.906195 24803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:06.906209 24803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:06.906222 24803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:38:06.906234 24803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:38:06.907698 24803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:33733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:37255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:37255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:06.907896 24803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:06.908082 24803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:06.913094 24815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:06.913097 24812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:06.913323 24810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:06.913224 24803 server_base.cc:1034] running on GCE node
I20260811 14:38:06.913622 24803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:06.913956 24803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:06.915186 24803 hybrid_clock.cc:648] HybridClock initialized: now 1786459086915165 us; error 52 us; skew 500 ppm
Aug 11 14:38:06 dist-test-slave-twwt krb5kdc[24784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459086, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:06.918118 24803 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:38:06.918586 24803 webserver.cc:460] Webserver started at http://127.2.103.126:44579/ using document root <none> and password file <none>
I20260811 14:38:06.918902 24803 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:06.918953 24803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:06.919054 24803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:38:06.920234 24803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "5ef19653a6824a399fadac561ec8619f"
format_stamp: "Formatted at 2026-08-11 14:38:06 on dist-test-slave-twwt"
server_key: "d3e6ad44ad59874ab32e7e1d0a2abf95"
server_key_iv: "f8140d7be61cf7b9ee4c1341c7d83220"
server_key_version: "encryptionkey@0"
I20260811 14:38:06.920581 24803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "5ef19653a6824a399fadac561ec8619f"
format_stamp: "Formatted at 2026-08-11 14:38:06 on dist-test-slave-twwt"
server_key: "d3e6ad44ad59874ab32e7e1d0a2abf95"
server_key_iv: "f8140d7be61cf7b9ee4c1341c7d83220"
server_key_version: "encryptionkey@0"
I20260811 14:38:06.921978 24803 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:38:06.922771 24823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:06.922938 24803 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:06.923032 24803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "5ef19653a6824a399fadac561ec8619f"
format_stamp: "Formatted at 2026-08-11 14:38:06 on dist-test-slave-twwt"
server_key: "d3e6ad44ad59874ab32e7e1d0a2abf95"
server_key_iv: "f8140d7be61cf7b9ee4c1341c7d83220"
server_key_version: "encryptionkey@0"
I20260811 14:38:06.923094 24803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:06.934468 24803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:06.936466 24803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:06.936576 24803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:06.940651 24803 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:37255
I20260811 14:38:06.940742 24918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:37255 every 8 connection(s)
I20260811 14:38:06.941108 24803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:38:06.942044 24920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:06.944554 24920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f: Bootstrap starting.
I20260811 14:38:06.944641 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 24803
I20260811 14:38:06.944775 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:38:06.944990 2461 external_mini_cluster.cc:1428] Setting key f9cc876e8773ad6099045437200095bf
I20260811 14:38:06.945405 24920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:06.945772 24920 log.cc:824] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:06.946593 24920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f: No bootstrap required, opened a new log
I20260811 14:38:06.949030 24920 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ef19653a6824a399fadac561ec8619f" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 37255 } }
I20260811 14:38:06.949169 24920 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:06.949193 24920 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ef19653a6824a399fadac561ec8619f, State: Initialized, Role: FOLLOWER
I20260811 14:38:06.949334 24920 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ef19653a6824a399fadac561ec8619f" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 37255 } }
I20260811 14:38:06.949416 24920 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:38:06.949440 24920 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:38:06.949509 24920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:38:06.950213 24920 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ef19653a6824a399fadac561ec8619f" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 37255 } }
I20260811 14:38:06.950331 24920 leader_election.cc:302] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ef19653a6824a399fadac561ec8619f; no voters:
I20260811 14:38:06.950503 24920 leader_election.cc:288] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:38:06.950628 24927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:38:06.950814 24927 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f [term 1 LEADER]: Becoming Leader. State: Replica: 5ef19653a6824a399fadac561ec8619f, State: Running, Role: LEADER
I20260811 14:38:06.950793 24920 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:38:06.950937 24927 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ef19653a6824a399fadac561ec8619f" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 37255 } }
I20260811 14:38:06.951329 24927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ef19653a6824a399fadac561ec8619f. Latest consensus state: current_term: 1 leader_uuid: "5ef19653a6824a399fadac561ec8619f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ef19653a6824a399fadac561ec8619f" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 37255 } } }
I20260811 14:38:06.951337 24930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ef19653a6824a399fadac561ec8619f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ef19653a6824a399fadac561ec8619f" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 37255 } } }
I20260811 14:38:06.951435 24927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f [sys.catalog]: This master's current role is: LEADER
I20260811 14:38:06.951457 24930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f [sys.catalog]: This master's current role is: LEADER
I20260811 14:38:06.952364 24937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 11 14:38:06 dist-test-slave-twwt krb5kdc[24784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:38:06.952900 24937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:38:06.954669 24937 catalog_manager.cc:1347] Generated new cluster ID: 825b9f3d2f994f20b67312d0d5d1f312
I20260811 14:38:06.954732 24937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:38:06.957979 24924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:06.945559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43674 (local address 127.2.103.126:37255)
0811 14:38:06.945763 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:06.945773 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:06.945816 (+ 43us) server_negotiation.cc:407] Connection header received
0811 14:38:06.946418 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:06.946429 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:06.946491 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:06.946678 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:06.947337 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:06.948845 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:06.949815 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:06.950453 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:06.950570 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:06.953726 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:06.953730 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:06.953731 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:06.953743 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:06.955932 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:06.956349 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:06.956352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:06.956352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:06.956459 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:06.956677 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:06.956678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:06.956678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:06.956997 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:06.957163 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:06.957611 (+ 448us) server_negotiation.cc:299] Negotiation successful
0811 14:38:06.957749 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":90,"threads_started":1}
I20260811 14:38:06.959254 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:38:06.973515 24937 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:38:06.974095 24937 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:38:06.980896 24937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ef19653a6824a399fadac561ec8619f: Generated new TSK 0
I20260811 14:38:06.981130 24937 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:38:07.013944 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 24803
2026-08-11T14:38:07Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1618 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:38:07 dist-test-slave-twwt krb5kdc[24963](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:38:07 dist-test-slave-twwt krb5kdc[24963](info): set up 2 sockets
Aug 11 14:38:07 dist-test-slave-twwt krb5kdc[24963](info): commencing operation
krb5kdc: starting...
W20260811 14:38:08.289695 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:38:08 dist-test-slave-twwt krb5kdc[24963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:38:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:38:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:38:08.419744 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:46077
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:46077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:38:08.505712 24984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:08.505964 24984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:08.506002 24984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:08.507169 24984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:38:08.507247 24984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:08.507273 24984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:08.507301 24984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:38:08.507323 24984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:38:08.508836 24984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:46077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:46077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:08.509123 24984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:08.509364 24984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:08.514382 24994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:08.514451 24992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:08.514618 24984 server_base.cc:1034] running on GCE node
W20260811 14:38:08.514398 24990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:08.514994 24984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:08.515323 24984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:08.516518 24984 hybrid_clock.cc:648] HybridClock initialized: now 1786459088516504 us; error 48 us; skew 500 ppm
Aug 11 14:38:08 dist-test-slave-twwt krb5kdc[24963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459088, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:08.519227 24984 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:38:08.519711 24984 webserver.cc:460] Webserver started at http://127.2.103.126:32981/ using document root <none> and password file <none>
I20260811 14:38:08.519937 24984 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:08.519979 24984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:08.520072 24984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:38:08.521122 24984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "15a67e27de6440389dc1877bcec288bb"
format_stamp: "Formatted at 2026-08-11 14:38:08 on dist-test-slave-twwt"
server_key: "fc713927050ed3c69d37291d769b23c4"
server_key_iv: "70a0640f68f641bf336fb52135d056fc"
server_key_version: "encryptionkey@0"
I20260811 14:38:08.521462 24984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "15a67e27de6440389dc1877bcec288bb"
format_stamp: "Formatted at 2026-08-11 14:38:08 on dist-test-slave-twwt"
server_key: "fc713927050ed3c69d37291d769b23c4"
server_key_iv: "70a0640f68f641bf336fb52135d056fc"
server_key_version: "encryptionkey@0"
I20260811 14:38:08.522786 24984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 14:38:08.523521 25005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:08.523710 24984 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:08.523797 24984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "15a67e27de6440389dc1877bcec288bb"
format_stamp: "Formatted at 2026-08-11 14:38:08 on dist-test-slave-twwt"
server_key: "fc713927050ed3c69d37291d769b23c4"
server_key_iv: "70a0640f68f641bf336fb52135d056fc"
server_key_version: "encryptionkey@0"
I20260811 14:38:08.523877 24984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:08.568468 24984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:08.570932 24984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:08.571099 24984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:08.575490 24984 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:46077
I20260811 14:38:08.575584 25080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:46077 every 8 connection(s)
I20260811 14:38:08.575966 24984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:38:08.576268 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 24984
I20260811 14:38:08.576396 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:38:08.576562 2461 external_mini_cluster.cc:1428] Setting key d65b130d2f24f9ecb71d03375cb109ee
I20260811 14:38:08.577256 25081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:08.580013 25081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb: Bootstrap starting.
I20260811 14:38:08.580683 25081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:08.581175 25081 log.cc:824] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:08.582032 25081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb: No bootstrap required, opened a new log
Aug 11 14:38:08 dist-test-slave-twwt krb5kdc[24963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:38:08.584332 25081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a67e27de6440389dc1877bcec288bb" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46077 } }
I20260811 14:38:08.584523 25081 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:08.584604 25081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15a67e27de6440389dc1877bcec288bb, State: Initialized, Role: FOLLOWER
I20260811 14:38:08.584795 25081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a67e27de6440389dc1877bcec288bb" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46077 } }
I20260811 14:38:08.584903 25081 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:38:08.584949 25081 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:38:08.585000 25081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:38:08.585577 25081 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a67e27de6440389dc1877bcec288bb" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46077 } }
I20260811 14:38:08.585701 25081 leader_election.cc:302] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15a67e27de6440389dc1877bcec288bb; no voters:
I20260811 14:38:08.585906 25081 leader_election.cc:288] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:38:08.585966 25090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:38:08.586182 25081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:38:08.586186 25090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 1 LEADER]: Becoming Leader. State: Replica: 15a67e27de6440389dc1877bcec288bb, State: Running, Role: LEADER
I20260811 14:38:08.586344 25090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a67e27de6440389dc1877bcec288bb" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46077 } }
I20260811 14:38:08.586798 25092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15a67e27de6440389dc1877bcec288bb. Latest consensus state: current_term: 1 leader_uuid: "15a67e27de6440389dc1877bcec288bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a67e27de6440389dc1877bcec288bb" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46077 } } }
I20260811 14:38:08.586966 25092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [sys.catalog]: This master's current role is: LEADER
I20260811 14:38:08.587249 25091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15a67e27de6440389dc1877bcec288bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a67e27de6440389dc1877bcec288bb" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46077 } } }
I20260811 14:38:08.587415 25091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [sys.catalog]: This master's current role is: LEADER
I20260811 14:38:08.587598 25096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:38:08.588160 25096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:38:08.588781 25088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:08.577116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58162 (local address 127.2.103.126:46077)
0811 14:38:08.577331 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:08.577343 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:08.577386 (+ 43us) server_negotiation.cc:407] Connection header received
0811 14:38:08.577879 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:08.577887 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:08.577938 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:08.578099 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:08.578837 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:08.579748 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:08.580628 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:08.581193 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:08.581337 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:08.584040 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:08.584045 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:08.584045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:08.584060 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:08.586360 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:08.586856 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:08.586861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:08.586861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:08.586979 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:08.587204 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:08.587205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:08.587205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:08.587611 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:08.587782 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:08.588494 (+ 712us) server_negotiation.cc:299] Negotiation successful
0811 14:38:08.588625 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":96,"threads_started":1}
I20260811 14:38:08.590081 25096 catalog_manager.cc:1347] Generated new cluster ID: 85d8e8292529415ab265ea17ab5acacd
I20260811 14:38:08.590157 25096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:38:08.593683 25096 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:38:08.594410 25096 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:38:08.609959 25096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb: Generated new TSK 0
I20260811 14:38:08.610230 25096 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
I20260811 14:38:08.653920 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:46077
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:38:08.751618 25117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:08.751844 25117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:08.751870 25117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:08.753098 25117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:08.753152 25117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:08.753216 25117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:38:08.754755 25117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:46077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:08.755028 25117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:08.755296 25117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:08.760618 25127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:08.760618 25126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:08.760838 25117 server_base.cc:1034] running on GCE node
W20260811 14:38:08.760747 25130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:08.761195 25117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:08.761428 25117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:08.762651 25117 hybrid_clock.cc:648] HybridClock initialized: now 1786459088762569 us; error 47 us; skew 500 ppm
Aug 11 14:38:08 dist-test-slave-twwt krb5kdc[24963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459088, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:08.769008 25117 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:38:08.769523 25117 webserver.cc:460] Webserver started at http://127.2.103.65:46355/ using document root <none> and password file <none>
I20260811 14:38:08.769789 25117 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:08.769840 25117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:08.770018 25117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:38:08.771250 25117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "56f4d08c06f146a89480fa9e5a591c2d"
format_stamp: "Formatted at 2026-08-11 14:38:08 on dist-test-slave-twwt"
server_key: "3cf50b218126343ca6835b982cac5fc5"
server_key_iv: "f332a75a58b7a499c09cd08c1af57ee2"
server_key_version: "encryptionkey@0"
I20260811 14:38:08.771627 25117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "56f4d08c06f146a89480fa9e5a591c2d"
format_stamp: "Formatted at 2026-08-11 14:38:08 on dist-test-slave-twwt"
server_key: "3cf50b218126343ca6835b982cac5fc5"
server_key_iv: "f332a75a58b7a499c09cd08c1af57ee2"
server_key_version: "encryptionkey@0"
I20260811 14:38:08.773072 25117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:38:08.773865 25143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:08.774077 25117 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:08.774184 25117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "56f4d08c06f146a89480fa9e5a591c2d"
format_stamp: "Formatted at 2026-08-11 14:38:08 on dist-test-slave-twwt"
server_key: "3cf50b218126343ca6835b982cac5fc5"
server_key_iv: "f332a75a58b7a499c09cd08c1af57ee2"
server_key_version: "encryptionkey@0"
I20260811 14:38:08.774262 25117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:08.787691 25117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:08.789959 25117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:08.790130 25117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:08.790400 25117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:38:08.790781 25117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:38:08.790843 25117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:08.790879 25117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:38:08.790915 25117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:08.797158 25117 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:35451
I20260811 14:38:08.797212 25329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:35451 every 8 connection(s)
I20260811 14:38:08.797557 25117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:38:08.800729 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 25117
I20260811 14:38:08.800864 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:38:08.801086 2461 external_mini_cluster.cc:1428] Setting key 16df210bab0c1e168ca971b2068675ef
Aug 11 14:38:08 dist-test-slave-twwt krb5kdc[24963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459088, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:38:08.808233 25332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:08.798272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.65:49687)
0811 14:38:08.798474 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:08.798520 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:08.799059 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:08.799311 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:08.799315 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:08.799914 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:08.800114 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:08.800119 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:08.800840 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:08.800842 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:08.801859 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:08.801863 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:08.802252 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:08.802253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:08.802420 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:08.803002 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:08.803014 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:08.804929 (+ 1915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:08.806856 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:08.806864 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:08.806866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:08.807097 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:08.807310 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:08.807311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:08.807311 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:08.807419 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:08.807720 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:08.807725 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:08.807845 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 14:38:08.807932 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 14:38:08.808064 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":116,"threads_started":1}
I20260811 14:38:08.808228 25088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:08.798307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:49687 (local address 127.2.103.126:46077)
0811 14:38:08.798399 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:08.798401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:08.798599 (+ 198us) server_negotiation.cc:407] Connection header received
0811 14:38:08.799151 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:08.799153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:08.799163 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:08.799225 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:08.800197 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:08.800733 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:08.801963 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:08.802158 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:08.802214 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:08.805041 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:08.805043 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:08.805043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:08.805053 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:08.806777 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:08.807163 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:08.807164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:08.807165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:08.807235 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:08.807490 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:08.807492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:08.807493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:08.807648 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:08.807726 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:08.807987 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 14:38:08.808087 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 14:38:08.808910 25330 heartbeater.cc:347] Connected to a master server at 127.2.103.126:46077
I20260811 14:38:08.809036 25330 heartbeater.cc:464] Registering TS with master...
I20260811 14:38:08.809269 25330 heartbeater.cc:511] Master 127.2.103.126:46077 requested a full tablet report, sending...
I20260811 14:38:08.809846 25027 ts_manager.cc:194] Registered new tserver with Master: 56f4d08c06f146a89480fa9e5a591c2d (127.2.103.65:35451)
I20260811 14:38:08.810473 25027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:49687
WARNING: no policy specified for kudu/127.2.103.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
I20260811 14:38:08.829849 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:46077
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:38:08.916005 25341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:08.916213 25341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:08.916236 25341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:08.917297 25341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:08.917332 25341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:08.917357 25341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:38:08.918834 25341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:46077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:08.919023 25341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:08.919216 25341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:08.924254 25351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:08.924243 25353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:08.924320 25341 server_base.cc:1034] running on GCE node
W20260811 14:38:08.924244 25349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:08.924690 25341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:08.924968 25341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:08.926148 25341 hybrid_clock.cc:648] HybridClock initialized: now 1786459088926132 us; error 48 us; skew 500 ppm
Aug 11 14:38:08 dist-test-slave-twwt krb5kdc[24963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459088, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:08.928848 25341 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:38:08.929356 25341 webserver.cc:460] Webserver started at http://127.2.103.66:40787/ using document root <none> and password file <none>
I20260811 14:38:08.929595 25341 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:08.929664 25341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:08.929824 25341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:38:08.931003 25341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "ba71ee3655164ecba3f4063134037551"
format_stamp: "Formatted at 2026-08-11 14:38:08 on dist-test-slave-twwt"
server_key: "9e75505cb97a1336f6268c97cc5ebbce"
server_key_iv: "ed753235864df63dd4ce5e6519f83353"
server_key_version: "encryptionkey@0"
I20260811 14:38:08.931384 25341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "ba71ee3655164ecba3f4063134037551"
format_stamp: "Formatted at 2026-08-11 14:38:08 on dist-test-slave-twwt"
server_key: "9e75505cb97a1336f6268c97cc5ebbce"
server_key_iv: "ed753235864df63dd4ce5e6519f83353"
server_key_version: "encryptionkey@0"
I20260811 14:38:08.932832 25341 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:38:08.933648 25364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:08.933823 25341 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:38:08.933950 25341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "ba71ee3655164ecba3f4063134037551"
format_stamp: "Formatted at 2026-08-11 14:38:08 on dist-test-slave-twwt"
server_key: "9e75505cb97a1336f6268c97cc5ebbce"
server_key_iv: "ed753235864df63dd4ce5e6519f83353"
server_key_version: "encryptionkey@0"
I20260811 14:38:08.934029 25341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:08.945622 25341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:08.948125 25341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:08.948307 25341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:08.948552 25341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:38:08.948902 25341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:38:08.948961 25341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:08.949004 25341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:38:08.949033 25341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:08.955185 25341 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:46803
I20260811 14:38:08.955240 25546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:46803 every 8 connection(s)
I20260811 14:38:08.955606 25341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:38:08.955726 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 25341
I20260811 14:38:08.955848 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:38:08.956101 2461 external_mini_cluster.cc:1428] Setting key b45f7a769350391cdc0ca6bde67491e4
Aug 11 14:38:08 dist-test-slave-twwt krb5kdc[24963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459088, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.103.67@KRBTEST.COM; defaulting to no policy
I20260811 14:38:08.966420 25088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:08.956527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:53267 (local address 127.2.103.126:46077)
0811 14:38:08.956613 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:08.956615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:08.956803 (+ 188us) server_negotiation.cc:407] Connection header received
0811 14:38:08.957415 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:08.957417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:08.957429 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:08.957496 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:08.958440 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:08.959117 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:08.960612 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:08.960794 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:08.960845 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:08.963453 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:08.963455 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:08.963455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:08.963464 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:08.964951 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:08.965395 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:08.965397 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:08.965397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:08.965461 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:08.965738 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:08.965739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:08.965739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:08.965886 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:08.965951 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:08.966265 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 14:38:08.966342 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 14:38:08.966579 25553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:08.956458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.66:53267)
0811 14:38:08.956649 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:08.956698 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:08.957298 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:08.957553 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:08.957556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:08.958145 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:08.958344 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:08.958349 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:08.959208 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:08.959209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:08.960521 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:08.960524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:08.960877 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:08.960879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:08.961037 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:08.961585 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:08.961597 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:08.963346 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:08.965056 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:08.965065 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:08.965067 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:08.965310 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:08.965538 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:08.965539 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:08.965539 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:08.965660 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:08.965990 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:08.965996 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:08.966121 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 14:38:08.966209 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 14:38:08.966344 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":99,"threads_started":1}
I20260811 14:38:08.967079 25551 heartbeater.cc:347] Connected to a master server at 127.2.103.126:46077
I20260811 14:38:08.967176 25551 heartbeater.cc:464] Registering TS with master...
I20260811 14:38:08.967334 25551 heartbeater.cc:511] Master 127.2.103.126:46077 requested a full tablet report, sending...
I20260811 14:38:08.967691 25027 ts_manager.cc:194] Registered new tserver with Master: ba71ee3655164ecba3f4063134037551 (127.2.103.66:46803)
I20260811 14:38:08.968087 25027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:53267
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
I20260811 14:38:08.986474 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:46077
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:38:09.073179 25556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:09.073427 25556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:09.073467 25556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:09.074661 25556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:09.074728 25556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:09.074775 25556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:38:09.076278 25556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:46077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:09.076592 25556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:09.076838 25556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:09.081977 25569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:09.082010 25568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:09.081983 25571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:09.082314 25556 server_base.cc:1034] running on GCE node
I20260811 14:38:09.082551 25556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:09.082842 25556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:09.084017 25556 hybrid_clock.cc:648] HybridClock initialized: now 1786459089084005 us; error 41 us; skew 500 ppm
Aug 11 14:38:09 dist-test-slave-twwt krb5kdc[24963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459089, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:09.086760 25556 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:38:09.087260 25556 webserver.cc:460] Webserver started at http://127.2.103.67:42505/ using document root <none> and password file <none>
I20260811 14:38:09.087554 25556 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:09.087626 25556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:09.087800 25556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:38:09.088934 25556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "6aad9e62431c49b9b115d08925eae94e"
format_stamp: "Formatted at 2026-08-11 14:38:09 on dist-test-slave-twwt"
server_key: "0c31573b8791ff1c85d97464d4420974"
server_key_iv: "fb748f6ab2a48565586a6ef29eeaad44"
server_key_version: "encryptionkey@0"
I20260811 14:38:09.089250 25556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "6aad9e62431c49b9b115d08925eae94e"
format_stamp: "Formatted at 2026-08-11 14:38:09 on dist-test-slave-twwt"
server_key: "0c31573b8791ff1c85d97464d4420974"
server_key_iv: "fb748f6ab2a48565586a6ef29eeaad44"
server_key_version: "encryptionkey@0"
I20260811 14:38:09.090535 25556 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:38:09.091359 25583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:09.091569 25556 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:09.091662 25556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "6aad9e62431c49b9b115d08925eae94e"
format_stamp: "Formatted at 2026-08-11 14:38:09 on dist-test-slave-twwt"
server_key: "0c31573b8791ff1c85d97464d4420974"
server_key_iv: "fb748f6ab2a48565586a6ef29eeaad44"
server_key_version: "encryptionkey@0"
I20260811 14:38:09.091743 25556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:09.118135 25556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:09.120561 25556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:09.120712 25556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:09.120985 25556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:38:09.121336 25556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:38:09.121398 25556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:09.121434 25556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:38:09.121461 25556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:09.127595 25556 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:37569
I20260811 14:38:09.127640 25791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:37569 every 8 connection(s)
I20260811 14:38:09.128055 25556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:38:09.133458 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 25556
I20260811 14:38:09.133586 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:38:09.133772 2461 external_mini_cluster.cc:1428] Setting key 261b7d11adbbd536aff35e4efe68235e
Aug 11 14:38:09 dist-test-slave-twwt krb5kdc[24963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459089, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:38:09.138561 25088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.128893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:42917 (local address 127.2.103.126:46077)
0811 14:38:09.128943 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:09.128945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:09.129232 (+ 287us) server_negotiation.cc:407] Connection header received
0811 14:38:09.129829 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:09.129831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:09.129847 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:09.129895 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:09.131032 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.131543 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.132631 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.132841 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.132922 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.135611 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:09.135613 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:09.135614 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:09.135622 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:09.137023 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:09.137459 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:09.137461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:09.137461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:09.137528 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:09.137805 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:09.137806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:09.137806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:09.137934 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:09.138014 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:09.138342 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 14:38:09.138467 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 14:38:09.138785 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.128898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.67:42917)
0811 14:38:09.129091 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:09.129136 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:09.129704 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:09.129972 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:09.129977 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:09.130679 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:09.130916 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.130922 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.131663 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.131664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.132520 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.132522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.132934 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.132935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.133074 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.133767 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:09.133777 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:09.135506 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:09.137115 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:09.137123 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:09.137125 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:09.137377 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:09.137600 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:09.137601 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:09.137602 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:09.137731 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:09.138044 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:09.138051 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:09.138197 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 14:38:09.138309 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 14:38:09.138492 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":98,"threads_started":1}
I20260811 14:38:09.139276 25792 heartbeater.cc:347] Connected to a master server at 127.2.103.126:46077
I20260811 14:38:09.139381 25792 heartbeater.cc:464] Registering TS with master...
I20260811 14:38:09.139547 25792 heartbeater.cc:511] Master 127.2.103.126:46077 requested a full tablet report, sending...
I20260811 14:38:09.139890 25027 ts_manager.cc:194] Registered new tserver with Master: 6aad9e62431c49b9b115d08925eae94e (127.2.103.67:37569)
I20260811 14:38:09.140229 25027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:42917
I20260811 14:38:09.145776 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:38:09.153487 25088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.147252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58172 (local address 127.2.103.126:46077)
0811 14:38:09.147322 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:09.147325 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:09.147448 (+ 123us) server_negotiation.cc:407] Connection header received
0811 14:38:09.147533 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:09.147534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:09.147550 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:09.147613 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:09.148269 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.148753 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.149520 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.149714 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.149790 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.151064 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:09.151066 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:09.151066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:09.151074 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:09.152274 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:09.152628 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:09.152629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:09.152630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:09.152693 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:09.152909 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:09.152910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:09.152910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:09.153069 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:09.153188 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:09.153297 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 14:38:09.153379 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 14:38:09.155331 25027 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:38:09.157037 25027 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:38:09.164386 25823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.160397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54410 (local address 127.2.103.65:35451)
0811 14:38:09.160593 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:09.160598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:09.160611 (+ 13us) server_negotiation.cc:407] Connection header received
0811 14:38:09.160656 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:09.160659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:09.160678 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:09.160816 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:09.160946 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.161709 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.163248 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.163988 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.164089 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.164153 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:09.164251 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 14:38:09.164295 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":84,"threads_started":1}
I20260811 14:38:09.164427 25820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.159990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:35451 (local address 127.0.0.1:54410)
0811 14:38:09.160116 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:09.160126 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:09.160170 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:09.160813 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:09.160813 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:09.160815 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:09.160875 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.160876 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.161833 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.161834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.163152 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.163155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.164097 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.164098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.164169 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.164173 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:38:09.164233 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 14:38:09.164281 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":26,"thread_start_us":86,"threads_started":1}
I20260811 14:38:09.164472 25817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.159838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48542 (local address 127.2.103.66:46803)
0811 14:38:09.160047 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:09.160052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:09.160064 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:38:09.160112 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:09.160114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:09.160127 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:09.160485 (+ 358us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:09.160569 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.161667 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.163088 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.163875 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.163962 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.164032 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:09.164333 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 14:38:09.164380 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":137,"threads_started":1}
I20260811 14:38:09.164446 25816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.159728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:46803 (local address 127.0.0.1:48542)
0811 14:38:09.159862 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:09.159908 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:09.159979 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:09.160298 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:09.160301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:09.160304 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:09.160399 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.160402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.161786 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.161787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.162985 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.162988 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.163981 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.163982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.164075 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.164092 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 14:38:09.164165 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 14:38:09.164230 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":40,"thread_start_us":83,"threads_started":1}
I20260811 14:38:09.164934 25822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.160362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42072 (local address 127.2.103.67:37569)
0811 14:38:09.160560 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:09.160565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:09.160773 (+ 208us) server_negotiation.cc:407] Connection header received
0811 14:38:09.160820 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:09.160822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:09.160839 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:09.160920 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:09.161131 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.162175 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.163455 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.164003 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.164075 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.164135 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:09.164802 (+ 667us) server_negotiation.cc:299] Negotiation successful
0811 14:38:09.164854 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":109,"threads_started":1}
I20260811 14:38:09.165036 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.160182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:37569 (local address 127.0.0.1:42072)
0811 14:38:09.160642 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:09.160653 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:09.160712 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:09.160998 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:09.160999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:09.161001 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:09.161046 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.161047 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.162290 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.162291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.163364 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.163366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.164172 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.164172 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.164234 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.164237 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:38:09.164285 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 14:38:09.164327 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":68,"threads_started":1}
I20260811 14:38:09.165369 25216 tablet_service.cc:1467] Processing CreateTablet for tablet fcd16066b03f4f3b8cb39e20fd762b1b (DEFAULT_TABLE table=test-table [id=e4b0e4b40ab043a0b95339e0ca7b8a66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:38:09.165795 25216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcd16066b03f4f3b8cb39e20fd762b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:09.165688 25436 tablet_service.cc:1467] Processing CreateTablet for tablet fcd16066b03f4f3b8cb39e20fd762b1b (DEFAULT_TABLE table=test-table [id=e4b0e4b40ab043a0b95339e0ca7b8a66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:38:09.165834 25669 tablet_service.cc:1467] Processing CreateTablet for tablet fcd16066b03f4f3b8cb39e20fd762b1b (DEFAULT_TABLE table=test-table [id=e4b0e4b40ab043a0b95339e0ca7b8a66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:38:09.166023 25436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcd16066b03f4f3b8cb39e20fd762b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:09.166108 25669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcd16066b03f4f3b8cb39e20fd762b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:09.169032 25824 tablet_bootstrap.cc:492] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e: Bootstrap starting.
I20260811 14:38:09.169538 25825 tablet_bootstrap.cc:492] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551: Bootstrap starting.
I20260811 14:38:09.170058 25824 tablet_bootstrap.cc:654] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:09.170266 25825 tablet_bootstrap.cc:654] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:09.170781 25827 tablet_bootstrap.cc:492] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d: Bootstrap starting.
I20260811 14:38:09.170776 25824 log.cc:824] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:09.170869 25825 log.cc:824] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:09.171476 25827 tablet_bootstrap.cc:654] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:09.171726 25824 tablet_bootstrap.cc:492] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e: No bootstrap required, opened a new log
I20260811 14:38:09.171789 25824 ts_tablet_manager.cc:1397] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 14:38:09.172077 25827 log.cc:824] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:09.172856 25827 tablet_bootstrap.cc:492] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d: No bootstrap required, opened a new log
I20260811 14:38:09.172971 25827 ts_tablet_manager.cc:1397] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:38:09.173578 25824 raft_consensus.cc:348] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } }
I20260811 14:38:09.173731 25824 raft_consensus.cc:374] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:09.173754 25824 raft_consensus.cc:729] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6aad9e62431c49b9b115d08925eae94e, State: Initialized, Role: FOLLOWER
I20260811 14:38:09.173887 25824 consensus_queue.cc:260] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } }
I20260811 14:38:09.174225 25824 ts_tablet_manager.cc:1428] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 14:38:09.174254 25792 heartbeater.cc:503] Master 127.2.103.126:46077 was elected leader, sending a full tablet report...
I20260811 14:38:09.174873 25825 tablet_bootstrap.cc:492] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551: No bootstrap required, opened a new log
I20260811 14:38:09.174938 25825 ts_tablet_manager.cc:1397] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260811 14:38:09.174853 25827 raft_consensus.cc:348] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } }
I20260811 14:38:09.175053 25827 raft_consensus.cc:374] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:09.175102 25827 raft_consensus.cc:729] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56f4d08c06f146a89480fa9e5a591c2d, State: Initialized, Role: FOLLOWER
I20260811 14:38:09.175230 25827 consensus_queue.cc:260] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } }
I20260811 14:38:09.175526 25827 ts_tablet_manager.cc:1428] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 14:38:09.175568 25330 heartbeater.cc:503] Master 127.2.103.126:46077 was elected leader, sending a full tablet report...
I20260811 14:38:09.176959 25825 raft_consensus.cc:348] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } }
I20260811 14:38:09.177100 25825 raft_consensus.cc:374] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:09.177121 25825 raft_consensus.cc:729] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba71ee3655164ecba3f4063134037551, State: Initialized, Role: FOLLOWER
I20260811 14:38:09.177201 25825 consensus_queue.cc:260] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } }
I20260811 14:38:09.177448 25551 heartbeater.cc:503] Master 127.2.103.126:46077 was elected leader, sending a full tablet report...
I20260811 14:38:09.177469 25825 ts_tablet_manager.cc:1428] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 14:38:09.196588 25837 raft_consensus.cc:482] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:38:09.196759 25837 raft_consensus.cc:504] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } }
I20260811 14:38:09.197178 25837 leader_election.cc:288] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56f4d08c06f146a89480fa9e5a591c2d (127.2.103.65:35451), 6aad9e62431c49b9b115d08925eae94e (127.2.103.67:37569)
I20260811 14:38:09.198486 25835 raft_consensus.cc:482] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:38:09.198596 25835 raft_consensus.cc:504] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } }
I20260811 14:38:09.198984 25835 leader_election.cc:288] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba71ee3655164ecba3f4063134037551 (127.2.103.66:46803), 56f4d08c06f146a89480fa9e5a591c2d (127.2.103.65:35451)
I20260811 14:38:09.200855 25553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.197421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:35451 (local address 127.2.103.66:38907)
0811 14:38:09.197489 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:09.197503 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:09.197564 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:09.197745 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:09.197747 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:09.197749 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:09.197818 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.197820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.198566 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.198567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.199797 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.199800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.200595 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.200597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.200678 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.200682 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:38:09.200725 (+ 43us) client_negotiation.cc:240] Negotiation successful
0811 14:38:09.200766 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 14:38:09.201011 25822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.197713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:44891 (local address 127.2.103.67:37569)
0811 14:38:09.197762 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:09.197765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:09.197809 (+ 44us) server_negotiation.cc:407] Connection header received
0811 14:38:09.197876 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:09.197878 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:09.197887 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:09.197943 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:09.198096 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.198722 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.200088 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.200644 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.200690 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.200698 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:09.200910 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 14:38:09.200947 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":27}
I20260811 14:38:09.201167 25823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.197500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:38907 (local address 127.2.103.65:35451)
0811 14:38:09.197574 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:09.197577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:09.197596 (+ 19us) server_negotiation.cc:407] Connection header received
0811 14:38:09.197632 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:09.197633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:09.197645 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:09.197707 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:09.197908 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.198463 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.199889 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.200486 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.200555 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.200564 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:09.201045 (+ 481us) server_negotiation.cc:299] Negotiation successful
0811 14:38:09.201085 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 14:38:09.201229 25840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.197549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:37569 (local address 127.2.103.66:44891)
0811 14:38:09.197756 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:09.197767 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:09.197823 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:09.197945 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:09.197946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:09.197947 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:09.198018 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.198020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.198825 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.198826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.199993 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.199995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.200759 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.200760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.200833 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.200836 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:38:09.201115 (+ 279us) client_negotiation.cc:240] Negotiation successful
0811 14:38:09.201157 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":126,"threads_started":1}
I20260811 14:38:09.201563 25710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcd16066b03f4f3b8cb39e20fd762b1b" candidate_uuid: "ba71ee3655164ecba3f4063134037551" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6aad9e62431c49b9b115d08925eae94e" is_pre_election: true
I20260811 14:38:09.201574 25254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcd16066b03f4f3b8cb39e20fd762b1b" candidate_uuid: "ba71ee3655164ecba3f4063134037551" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56f4d08c06f146a89480fa9e5a591c2d" is_pre_election: true
I20260811 14:38:09.201679 25710 raft_consensus.cc:2454] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba71ee3655164ecba3f4063134037551 in term 0.
I20260811 14:38:09.201735 25254 raft_consensus.cc:2454] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba71ee3655164ecba3f4063134037551 in term 0.
I20260811 14:38:09.201982 25368 leader_election.cc:302] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56f4d08c06f146a89480fa9e5a591c2d, ba71ee3655164ecba3f4063134037551; no voters:
I20260811 14:38:09.202167 25837 raft_consensus.cc:2781] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:38:09.202215 25837 raft_consensus.cc:482] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:38:09.202236 25837 raft_consensus.cc:3037] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:38:09.202661 25817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.199182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:48421 (local address 127.2.103.66:46803)
0811 14:38:09.199220 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:09.199222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:09.199374 (+ 152us) server_negotiation.cc:407] Connection header received
0811 14:38:09.199395 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:09.199395 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:09.199403 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:09.199444 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:09.199590 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.200251 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.201845 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.202316 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.202388 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.202396 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:09.202550 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 14:38:09.202583 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 14:38:09.202662 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.199113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:46803 (local address 127.2.103.67:48421)
0811 14:38:09.199265 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:09.199276 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:09.199324 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:09.199477 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:09.199478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:09.199479 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:09.199529 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.199530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.200332 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.200333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.201384 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.201387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.202389 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.202390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.202474 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.202477 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:38:09.202544 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 14:38:09.202583 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135}
I20260811 14:38:09.202967 25469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcd16066b03f4f3b8cb39e20fd762b1b" candidate_uuid: "6aad9e62431c49b9b115d08925eae94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba71ee3655164ecba3f4063134037551" is_pre_election: true
I20260811 14:38:09.203258 25837 raft_consensus.cc:504] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } }
I20260811 14:38:09.203388 25469 raft_consensus.cc:2379] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6aad9e62431c49b9b115d08925eae94e in current term 1: Already voted for candidate ba71ee3655164ecba3f4063134037551 in this term.
I20260811 14:38:09.203413 25837 leader_election.cc:288] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [CANDIDATE]: Term 1 election: Requested vote from peers 56f4d08c06f146a89480fa9e5a591c2d (127.2.103.65:35451), 6aad9e62431c49b9b115d08925eae94e (127.2.103.67:37569)
I20260811 14:38:09.203641 25842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.199224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:35451 (local address 127.2.103.67:52441)
0811 14:38:09.199658 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:09.199668 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:09.199723 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:09.199913 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:09.199914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:09.199915 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:09.199982 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.199984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.200832 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.200833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.202522 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.202525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.203366 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.203367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.203445 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.203448 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:38:09.203511 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 14:38:09.203553 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":89,"threads_started":1}
I20260811 14:38:09.203639 25254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcd16066b03f4f3b8cb39e20fd762b1b" candidate_uuid: "ba71ee3655164ecba3f4063134037551" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56f4d08c06f146a89480fa9e5a591c2d"
I20260811 14:38:09.203696 25710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcd16066b03f4f3b8cb39e20fd762b1b" candidate_uuid: "ba71ee3655164ecba3f4063134037551" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6aad9e62431c49b9b115d08925eae94e"
I20260811 14:38:09.203729 25254 raft_consensus.cc:3037] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:38:09.203761 25710 raft_consensus.cc:3037] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:38:09.204639 25254 raft_consensus.cc:2454] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba71ee3655164ecba3f4063134037551 in term 1.
I20260811 14:38:09.204639 25710 raft_consensus.cc:2454] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba71ee3655164ecba3f4063134037551 in term 1.
I20260811 14:38:09.203649 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.199472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:52441 (local address 127.2.103.65:35451)
0811 14:38:09.199609 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:09.199610 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:09.199775 (+ 165us) server_negotiation.cc:407] Connection header received
0811 14:38:09.199797 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:09.199798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:09.199807 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:09.199868 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:09.200053 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.200750 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.202647 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.203241 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.203330 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.203338 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:09.203520 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 14:38:09.203565 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260811 14:38:09.204850 25368 leader_election.cc:302] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56f4d08c06f146a89480fa9e5a591c2d, ba71ee3655164ecba3f4063134037551; no voters:
I20260811 14:38:09.204895 25254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcd16066b03f4f3b8cb39e20fd762b1b" candidate_uuid: "6aad9e62431c49b9b115d08925eae94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56f4d08c06f146a89480fa9e5a591c2d" is_pre_election: true
I20260811 14:38:09.204983 25254 raft_consensus.cc:2379] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6aad9e62431c49b9b115d08925eae94e in current term 1: Already voted for candidate ba71ee3655164ecba3f4063134037551 in this term.
I20260811 14:38:09.204985 25837 raft_consensus.cc:2781] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:38:09.205142 25837 raft_consensus.cc:686] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [term 1 LEADER]: Becoming Leader. State: Replica: ba71ee3655164ecba3f4063134037551, State: Running, Role: LEADER
I20260811 14:38:09.205228 25586 leader_election.cc:302] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6aad9e62431c49b9b115d08925eae94e; no voters: 56f4d08c06f146a89480fa9e5a591c2d, ba71ee3655164ecba3f4063134037551
I20260811 14:38:09.205237 25837 consensus_queue.cc:237] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } }
I20260811 14:38:09.205413 25835 raft_consensus.cc:2726] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 14:38:09.206136 25552 tablet.cc:2369] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:38:09.206125 25027 catalog_manager.cc:5495] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba71ee3655164ecba3f4063134037551 (127.2.103.66). New cstate: current_term: 1 leader_uuid: "ba71ee3655164ecba3f4063134037551" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } health_report { overall_health: UNKNOWN } } }
I20260811 14:38:09.212235 25817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.209860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48544 (local address 127.2.103.66:46803)
0811 14:38:09.209919 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:09.209921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:09.209930 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:38:09.209965 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:09.209966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:09.209983 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:09.210052 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:38:09.210162 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.210662 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.211422 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.211625 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.211701 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.211886 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:38:09.211985 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:38:09.212072 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:09.212117 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 14:38:09.212136 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 14:38:09.213943 25710 raft_consensus.cc:1261] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e [term 1 FOLLOWER]: Refusing update from remote peer ba71ee3655164ecba3f4063134037551: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:38:09.213976 25254 raft_consensus.cc:1261] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Refusing update from remote peer ba71ee3655164ecba3f4063134037551: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:38:09.214299 25837 consensus_queue.cc:1035] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:38:09.214432 25844 consensus_queue.cc:1035] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:38:09.217573 25851 mvcc.cc:205] Tried to move back new op lower bound from 7317336429417627648 to 7317336429385302016. Current Snapshot: MvccSnapshot[applied={T|T < 7317336429417627648}]
I20260811 14:38:09.217693 25852 mvcc.cc:205] Tried to move back new op lower bound from 7317336429417627648 to 7317336429385302016. Current Snapshot: MvccSnapshot[applied={T|T < 7317336429417627648}]
I20260811 14:38:09.219594 25026 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:38:09.219714 25026 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:38:09.220571 25026 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb: Sending DeleteTablet for 3 replicas of tablet fcd16066b03f4f3b8cb39e20fd762b1b
I20260811 14:38:09.220934 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 25117
I20260811 14:38:09.221002 25669 tablet_service.cc:1514] Processing DeleteTablet for tablet fcd16066b03f4f3b8cb39e20fd762b1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:38:09 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:42072
I20260811 14:38:09.221040 25436 tablet_service.cc:1514] Processing DeleteTablet for tablet fcd16066b03f4f3b8cb39e20fd762b1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:38:09 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:48542
I20260811 14:38:09.234789 25861 tablet_replica.cc:330] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551: stopping tablet replica
I20260811 14:38:09.235019 25861 raft_consensus.cc:2229] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:38:09.235193 25861 raft_consensus.cc:2258] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:38:09.235388 25860 tablet_replica.cc:330] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e: stopping tablet replica
I20260811 14:38:09.235515 25860 raft_consensus.cc:2229] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:38:09.235634 25860 raft_consensus.cc:2258] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:38:09.235960 25861 ts_tablet_manager.cc:1905] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:38:09.236133 25860 ts_tablet_manager.cc:1905] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:38:09.238266 25861 ts_tablet_manager.cc:1918] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:38:09.238289 25860 ts_tablet_manager.cc:1918] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:38:09.238361 25860 log.cc:1196] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/fcd16066b03f4f3b8cb39e20fd762b1b
I20260811 14:38:09.238497 25861 log.cc:1196] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/fcd16066b03f4f3b8cb39e20fd762b1b
I20260811 14:38:09.238689 25860 ts_tablet_manager.cc:1939] T fcd16066b03f4f3b8cb39e20fd762b1b P 6aad9e62431c49b9b115d08925eae94e: Deleting consensus metadata
I20260811 14:38:09.238996 25861 ts_tablet_manager.cc:1939] T fcd16066b03f4f3b8cb39e20fd762b1b P ba71ee3655164ecba3f4063134037551: Deleting consensus metadata
I20260811 14:38:09.239311 25008 catalog_manager.cc:4841] TS 6aad9e62431c49b9b115d08925eae94e (127.2.103.67:37569): tablet fcd16066b03f4f3b8cb39e20fd762b1b (table test-table [id=e4b0e4b40ab043a0b95339e0ca7b8a66]) successfully deleted
I20260811 14:38:09.239509 25008 catalog_manager.cc:4841] TS ba71ee3655164ecba3f4063134037551 (127.2.103.66:46803): tablet fcd16066b03f4f3b8cb39e20fd762b1b (table test-table [id=e4b0e4b40ab043a0b95339e0ca7b8a66]) successfully deleted
W20260811 14:38:09.239996 25008 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.103.65:35451 (error 108)
I20260811 14:38:09.240321 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 25341
I20260811 14:38:09.240515 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.240392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:35451 (local address 127.0.0.1:54422)
0811 14:38:09.240439 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:09.240477 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.65:35451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260811 14:38:09.244524 25008 catalog_manager.cc:4584] TS 56f4d08c06f146a89480fa9e5a591c2d (127.2.103.65:35451): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fcd16066b03f4f3b8cb39e20fd762b1b: Network error: Client connection negotiation failed: client connection to 127.2.103.65:35451: connect: Connection refused (error 111)
I20260811 14:38:09.253006 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 25556
I20260811 14:38:09.272151 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 24984
I20260811 14:38:09.282706 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:46077
--webserver_interface=127.2.103.126
--webserver_port=32981
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:46077
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:38:09.368850 25863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:09.369073 25863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:09.369099 25863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:09.370330 25863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:38:09.370388 25863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:09.370404 25863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:09.370415 25863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:38:09.370429 25863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:38:09.372051 25863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:46077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:46077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=32981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:09.372330 25863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:09.372608 25863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:09.382754 25879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:09.382712 25876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:09.382704 25872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:09.382714 25863 server_base.cc:1034] running on GCE node
I20260811 14:38:09.383338 25863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:09.383713 25863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:09.384887 25863 hybrid_clock.cc:648] HybridClock initialized: now 1786459089384855 us; error 46 us; skew 500 ppm
Aug 11 14:38:09 dist-test-slave-twwt krb5kdc[24963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459089, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:09.387676 25863 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:38:09.388247 25863 webserver.cc:460] Webserver started at http://127.2.103.126:32981/ using document root <none> and password file <none>
I20260811 14:38:09.388509 25863 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:09.388599 25863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:09.390189 25863 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:09.391137 25892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:09.391332 25863 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:38:09.391455 25863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "15a67e27de6440389dc1877bcec288bb"
format_stamp: "Formatted at 2026-08-11 14:38:08 on dist-test-slave-twwt"
server_key: "fc713927050ed3c69d37291d769b23c4"
server_key_iv: "70a0640f68f641bf336fb52135d056fc"
server_key_version: "encryptionkey@0"
I20260811 14:38:09.391942 25863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:09.419169 25863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:09.421232 25863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:09.421420 25863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:09.425999 25863 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:46077
I20260811 14:38:09.426039 25978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:46077 every 8 connection(s)
I20260811 14:38:09.426487 25863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:38:09.428493 25979 sys_catalog.cc:263] Verifying existing consensus state
I20260811 14:38:09.428874 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 25863
I20260811 14:38:09.429209 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:35451
--local_ip_for_outbound_sockets=127.2.103.65
--tserver_master_addrs=127.2.103.126:46077
--webserver_port=46355
--webserver_interface=127.2.103.65
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:38:09.429397 25979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb: Bootstrap starting.
I20260811 14:38:09.440492 25979 log.cc:824] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:09.443621 25979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:38:09.443994 25979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb: Bootstrap complete.
I20260811 14:38:09.447531 25979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a67e27de6440389dc1877bcec288bb" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46077 } }
I20260811 14:38:09.447894 25979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15a67e27de6440389dc1877bcec288bb, State: Initialized, Role: FOLLOWER
I20260811 14:38:09.448068 25979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a67e27de6440389dc1877bcec288bb" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46077 } }
I20260811 14:38:09.448137 25979 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:38:09.448189 25979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:38:09.448239 25979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 1 FOLLOWER]: Advancing to term 2
I20260811 14:38:09.449712 25979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a67e27de6440389dc1877bcec288bb" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46077 } }
I20260811 14:38:09.449838 25979 leader_election.cc:302] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15a67e27de6440389dc1877bcec288bb; no voters:
I20260811 14:38:09.450042 25979 leader_election.cc:288] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 14:38:09.450219 25991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 2 FOLLOWER]: Leader election won for term 2
I20260811 14:38:09.450306 25979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:38:09.450517 25991 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 2 LEADER]: Becoming Leader. State: Replica: 15a67e27de6440389dc1877bcec288bb, State: Running, Role: LEADER
I20260811 14:38:09.450724 25991 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a67e27de6440389dc1877bcec288bb" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46077 } }
I20260811 14:38:09.451112 25993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "15a67e27de6440389dc1877bcec288bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a67e27de6440389dc1877bcec288bb" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46077 } } }
I20260811 14:38:09.451205 25993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [sys.catalog]: This master's current role is: LEADER
I20260811 14:38:09.451246 25994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15a67e27de6440389dc1877bcec288bb. Latest consensus state: current_term: 2 leader_uuid: "15a67e27de6440389dc1877bcec288bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a67e27de6440389dc1877bcec288bb" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46077 } } }
I20260811 14:38:09.451306 25994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [sys.catalog]: This master's current role is: LEADER
I20260811 14:38:09.451454 26010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:38:09.452348 26010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:38:09.452556 26010 catalog_manager.cc:1259] Loaded cluster ID: 85d8e8292529415ab265ea17ab5acacd
I20260811 14:38:09.452831 26010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:38:09.453092 26010 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 14:38:09.455879 26010 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 14:38:09.457849 26010 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:38:09.458087 26010 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb: Loaded TSK: 0
I20260811 14:38:09.458323 26010 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 14:38:09.531369 25987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:09.531608 25987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:09.531646 25987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:09.532781 25987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:09.532850 25987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:09.532902 25987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:38:09.534392 25987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:35451
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=46355
--tserver_master_addrs=127.2.103.126:46077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:09.534711 25987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:09.534968 25987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:09.541584 26028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:09.541596 26031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:09.541570 26029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:09.541916 25987 server_base.cc:1034] running on GCE node
I20260811 14:38:09.542146 25987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:09.542452 25987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:09.543651 25987 hybrid_clock.cc:648] HybridClock initialized: now 1786459089543621 us; error 41 us; skew 500 ppm
Aug 11 14:38:09 dist-test-slave-twwt krb5kdc[24963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459089, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:09.546993 25987 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:38:09.547609 25987 webserver.cc:460] Webserver started at http://127.2.103.65:46355/ using document root <none> and password file <none>
I20260811 14:38:09.547915 25987 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:09.548009 25987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:09.549876 25987 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:38:09.550716 26042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:09.550925 25987 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:09.551043 25987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "56f4d08c06f146a89480fa9e5a591c2d"
format_stamp: "Formatted at 2026-08-11 14:38:08 on dist-test-slave-twwt"
server_key: "3cf50b218126343ca6835b982cac5fc5"
server_key_iv: "f332a75a58b7a499c09cd08c1af57ee2"
server_key_version: "encryptionkey@0"
I20260811 14:38:09.551484 25987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:09.567703 25987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:09.570088 25987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:09.570246 25987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:09.570524 25987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:38:09.571034 26052 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 14:38:09.573240 25987 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 14:38:09.573350 25987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 14:38:09.573406 25987 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 14:38:09.574155 25987 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 14:38:09.574222 25987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 14:38:09.574301 26052 tablet_bootstrap.cc:492] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d: Bootstrap starting.
I20260811 14:38:09.581887 25987 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:35451
I20260811 14:38:09.582324 25987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:38:09.585774 26052 log.cc:824] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:09.586342 26228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:35451 every 8 connection(s)
I20260811 14:38:09.586890 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 25987
I20260811 14:38:09.587190 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:46803
--local_ip_for_outbound_sockets=127.2.103.66
--tserver_master_addrs=127.2.103.126:46077
--webserver_port=40787
--webserver_interface=127.2.103.66
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:38:09.589481 26052 tablet_bootstrap.cc:492] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:38:09.589790 26052 tablet_bootstrap.cc:492] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d: Bootstrap complete.
I20260811 14:38:09.590870 26052 ts_tablet_manager.cc:1397] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.011s
I20260811 14:38:09.592310 26052 raft_consensus.cc:348] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } }
I20260811 14:38:09.592643 26052 raft_consensus.cc:729] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56f4d08c06f146a89480fa9e5a591c2d, State: Initialized, Role: FOLLOWER
I20260811 14:38:09.592813 26052 consensus_queue.cc:260] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } }
I20260811 14:38:09.593101 26052 ts_tablet_manager.cc:1428] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
Aug 11 14:38:09 dist-test-slave-twwt krb5kdc[24963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459089, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:38:09.599820 26235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.587014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.65:38039)
0811 14:38:09.587410 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:09.587456 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:09.588079 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:09.588767 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:09.588772 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:09.589141 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:09.589427 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.589434 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.591222 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.591223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.592482 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.592485 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.593543 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.593544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.593676 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.594248 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:09.594258 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:09.595833 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:09.597976 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:09.597985 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:09.597988 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:09.598227 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:09.598529 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:09.598530 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:09.598530 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:09.598720 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:09.599124 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:09.599132 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:09.599288 (+ 156us) client_negotiation.cc:769] Sending connection context
0811 14:38:09.599402 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 14:38:09.599600 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":86,"threads_started":1}
I20260811 14:38:09.599849 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.587066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:38039 (local address 127.2.103.126:46077)
0811 14:38:09.588303 (+ 1237us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:09.588313 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:09.588354 (+ 41us) server_negotiation.cc:407] Connection header received
0811 14:38:09.588529 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:09.588535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:09.588588 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:09.589568 (+ 980us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:09.589654 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.591118 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.592600 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.593433 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.593571 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.595972 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:09.595981 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:09.595981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:09.595993 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:09.597863 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:09.598342 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:09.598349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:09.598350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:09.598447 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:09.598787 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:09.598788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:09.598788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:09.599036 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:09.599131 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:09.599475 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 14:38:09.599607 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1182,"thread_start_us":88,"threads_started":1}
I20260811 14:38:09.601120 26229 heartbeater.cc:347] Connected to a master server at 127.2.103.126:46077
I20260811 14:38:09.601238 26229 heartbeater.cc:464] Registering TS with master...
I20260811 14:38:09.601449 26229 heartbeater.cc:511] Master 127.2.103.126:46077 requested a full tablet report, sending...
I20260811 14:38:09.602244 25920 ts_manager.cc:194] Registered new tserver with Master: 56f4d08c06f146a89480fa9e5a591c2d (127.2.103.65:35451)
I20260811 14:38:09.603469 25920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:38039
I20260811 14:38:09.604550 26229 heartbeater.cc:503] Master 127.2.103.126:46077 was elected leader, sending a full tablet report...
I20260811 14:38:09.609823 26242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.602933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:35451 (local address 127.0.0.1:54434)
0811 14:38:09.606734 (+ 3801us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:09.606768 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:09.606861 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:09.607062 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:09.607065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:09.607067 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:09.607134 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.607136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.607978 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.607979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.608804 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.608807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.609478 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.609479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.609566 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.609579 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 14:38:09.609684 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 14:38:09.609744 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3763,"thread_start_us":91,"threads_started":1}
I20260811 14:38:09.609823 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.603196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54434 (local address 127.2.103.65:35451)
0811 14:38:09.603409 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:09.603413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:09.606841 (+ 3428us) server_negotiation.cc:407] Connection header received
0811 14:38:09.606945 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:09.606948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:09.606962 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:09.607058 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:09.607204 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.607877 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.608902 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.609385 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.609478 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.609540 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:09.609703 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 14:38:09.609746 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":116,"threads_started":1}
I20260811 14:38:09.610549 26122 tablet_service.cc:1514] Processing DeleteTablet for tablet fcd16066b03f4f3b8cb39e20fd762b1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:38:09 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:54434
I20260811 14:38:09.611214 26245 tablet_replica.cc:330] stopping tablet replica
I20260811 14:38:09.611351 26245 raft_consensus.cc:2229] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:38:09.611438 26245 raft_consensus.cc:2258] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:38:09.611765 26245 ts_tablet_manager.cc:1905] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:38:09.613808 26245 ts_tablet_manager.cc:1918] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:38:09.613899 26245 log.cc:1196] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/wal/wals/fcd16066b03f4f3b8cb39e20fd762b1b
I20260811 14:38:09.614141 26245 ts_tablet_manager.cc:1939] T fcd16066b03f4f3b8cb39e20fd762b1b P 56f4d08c06f146a89480fa9e5a591c2d: Deleting consensus metadata
I20260811 14:38:09.614719 25895 catalog_manager.cc:4841] TS 56f4d08c06f146a89480fa9e5a591c2d (127.2.103.65:35451): tablet fcd16066b03f4f3b8cb39e20fd762b1b (table test-table [id=e4b0e4b40ab043a0b95339e0ca7b8a66]) successfully deleted
W20260811 14:38:09.688019 26240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:09.688228 26240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:09.688251 26240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:09.689349 26240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:09.689385 26240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:09.689411 26240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:38:09.690929 26240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:46803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=40787
--tserver_master_addrs=127.2.103.126:46077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:09.691138 26240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:09.691339 26240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:09.696638 26257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:09.696509 26259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:09.696547 26240 server_base.cc:1034] running on GCE node
W20260811 14:38:09.696539 26256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:09.697000 26240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:09.697265 26240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:09.698441 26240 hybrid_clock.cc:648] HybridClock initialized: now 1786459089698432 us; error 29 us; skew 500 ppm
Aug 11 14:38:09 dist-test-slave-twwt krb5kdc[24963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459089, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:09.701175 26240 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:38:09.701608 26240 webserver.cc:460] Webserver started at http://127.2.103.66:40787/ using document root <none> and password file <none>
I20260811 14:38:09.701825 26240 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:09.701891 26240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:09.703362 26240 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:09.703943 26276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:09.704139 26240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:09.704236 26240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "ba71ee3655164ecba3f4063134037551"
format_stamp: "Formatted at 2026-08-11 14:38:08 on dist-test-slave-twwt"
server_key: "9e75505cb97a1336f6268c97cc5ebbce"
server_key_iv: "ed753235864df63dd4ce5e6519f83353"
server_key_version: "encryptionkey@0"
I20260811 14:38:09.704587 26240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:09.726045 26240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:09.728125 26240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:09.728248 26240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:09.728471 26240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:38:09.728816 26240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:38:09.728849 26240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:09.728873 26240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:38:09.728925 26240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:09.734779 26240 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:46803
I20260811 14:38:09.734841 26492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:46803 every 8 connection(s)
I20260811 14:38:09.735203 26240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:38:09.737133 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 26240
I20260811 14:38:09.737419 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:37569
--local_ip_for_outbound_sockets=127.2.103.67
--tserver_master_addrs=127.2.103.126:46077
--webserver_port=42505
--webserver_interface=127.2.103.67
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 14:38:09 dist-test-slave-twwt krb5kdc[24963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459089, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:38:09.745643 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.735981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:54171 (local address 127.2.103.126:46077)
0811 14:38:09.736065 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:09.736068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:09.736274 (+ 206us) server_negotiation.cc:407] Connection header received
0811 14:38:09.736770 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:09.736771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:09.736785 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:09.736839 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:09.738004 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.738521 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.739584 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.739799 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.739888 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.742380 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:09.742382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:09.742383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:09.742393 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:09.744127 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:09.744563 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:09.744564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:09.744565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:09.744635 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:09.744889 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:09.744890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:09.744891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:09.745056 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:09.745139 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:09.745395 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 14:38:09.745503 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 14:38:09.745657 26495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.735957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.66:54171)
0811 14:38:09.736152 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:09.736190 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:09.736676 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:09.736935 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:09.736939 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:09.737663 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:09.737886 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.737899 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.738656 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.738657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.739479 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.739481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.739908 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.739909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.740061 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.740562 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:09.740571 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:09.742270 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:09.744217 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:09.744224 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:09.744227 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:09.744480 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:09.744712 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:09.744713 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:09.744713 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:09.744819 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:09.745131 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:09.745136 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:09.745258 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 14:38:09.745342 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 14:38:09.745484 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":115,"threads_started":1}
I20260811 14:38:09.746354 26493 heartbeater.cc:347] Connected to a master server at 127.2.103.126:46077
I20260811 14:38:09.746477 26493 heartbeater.cc:464] Registering TS with master...
I20260811 14:38:09.746750 26493 heartbeater.cc:511] Master 127.2.103.126:46077 requested a full tablet report, sending...
I20260811 14:38:09.747208 25920 ts_manager.cc:194] Registered new tserver with Master: ba71ee3655164ecba3f4063134037551 (127.2.103.66:46803)
I20260811 14:38:09.747593 25920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:54171
W20260811 14:38:09.826179 26498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:09.826404 26498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:09.826429 26498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:09.827641 26498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:09.827687 26498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:09.827718 26498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:38:09.829268 26498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:37569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=42505
--tserver_master_addrs=127.2.103.126:46077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:09.829528 26498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:09.829787 26498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:09.834954 26514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:09.834959 26519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:09.834961 26517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:09.835023 26498 server_base.cc:1034] running on GCE node
I20260811 14:38:09.835340 26498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:09.835646 26498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:09.836841 26498 hybrid_clock.cc:648] HybridClock initialized: now 1786459089836824 us; error 28 us; skew 500 ppm
Aug 11 14:38:09 dist-test-slave-twwt krb5kdc[24963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459089, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:09.839639 26498 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:38:09.840131 26498 webserver.cc:460] Webserver started at http://127.2.103.67:42505/ using document root <none> and password file <none>
I20260811 14:38:09.840397 26498 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:09.840497 26498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:09.841873 26498 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:38:09.842501 26528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:09.842733 26498 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:09.842841 26498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "6aad9e62431c49b9b115d08925eae94e"
format_stamp: "Formatted at 2026-08-11 14:38:09 on dist-test-slave-twwt"
server_key: "0c31573b8791ff1c85d97464d4420974"
server_key_iv: "fb748f6ab2a48565586a6ef29eeaad44"
server_key_version: "encryptionkey@0"
I20260811 14:38:09.843230 26498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:09.854580 26498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:09.857237 26498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:09.857365 26498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:09.857553 26498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:38:09.857808 26498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:38:09.857836 26498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:09.857859 26498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:38:09.857885 26498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:09.863875 26498 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:37569
I20260811 14:38:09.863952 26711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:37569 every 8 connection(s)
I20260811 14:38:09.864369 26498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
Aug 11 14:38:09 dist-test-slave-twwt krb5kdc[24963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459089, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:38:09.874081 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.865201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:46103 (local address 127.2.103.126:46077)
0811 14:38:09.865277 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:09.865279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:09.865507 (+ 228us) server_negotiation.cc:407] Connection header received
0811 14:38:09.866075 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:09.866077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:09.866091 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:09.866150 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:09.867240 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.867753 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.868786 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.869010 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.869093 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.871326 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:09.871328 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:09.871328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:09.871337 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:09.872611 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:09.873054 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:09.873055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:09.873056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:09.873123 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:09.873390 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:09.873391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:09.873391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:09.873517 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:09.873598 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:09.873883 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 14:38:09.873982 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 14:38:09.874437 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 26498
I20260811 14:38:09.874774 26717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.865193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.67:46103)
0811 14:38:09.865371 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:09.865416 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:09.865953 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:09.866234 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:09.866238 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:09.866912 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:09.867123 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.867135 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.867857 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.867859 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.868682 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.868685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.869102 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.869103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.869244 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.869741 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:09.869750 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:09.871217 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:09.872715 (+ 1498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:09.872724 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:09.872727 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:09.872970 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:09.873202 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:09.873203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:09.873204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:09.873319 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:09.873590 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:09.873597 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:09.873744 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 14:38:09.873860 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 14:38:09.874029 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":82,"threads_started":1}
I20260811 14:38:09.875236 26714 heartbeater.cc:347] Connected to a master server at 127.2.103.126:46077
I20260811 14:38:09.875351 26714 heartbeater.cc:464] Registering TS with master...
I20260811 14:38:09.875572 26714 heartbeater.cc:511] Master 127.2.103.126:46077 requested a full tablet report, sending...
I20260811 14:38:09.876013 25920 ts_manager.cc:194] Registered new tserver with Master: 6aad9e62431c49b9b115d08925eae94e (127.2.103.67:37569)
I20260811 14:38:09.876415 25920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:46103
I20260811 14:38:09.881453 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.874932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58176 (local address 127.2.103.126:46077)
0811 14:38:09.874975 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:09.874976 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:09.875231 (+ 255us) server_negotiation.cc:407] Connection header received
0811 14:38:09.875290 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:09.875291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:09.875302 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:09.875356 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:09.876114 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.876660 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.877452 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.877661 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.877737 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.878964 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:09.878966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:09.878966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:09.878974 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:09.880274 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:09.880646 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:09.880648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:09.880648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:09.880708 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:09.880920 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:09.880921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:09.880921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:09.881040 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:09.881227 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:09.881265 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 14:38:09.881348 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:38:09.882213 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:38:09.885231 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.882574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58192 (local address 127.2.103.126:46077)
0811 14:38:09.882670 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:09.882672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:09.882821 (+ 149us) server_negotiation.cc:407] Connection header received
0811 14:38:09.882931 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:09.882932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:09.882982 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:09.883052 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:38:09.883194 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.883660 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.884439 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.884646 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.884722 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.884896 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:38:09.885006 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:38:09.885095 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:09.885134 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 14:38:09.885151 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 14:38:09.885586 25920 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:58192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:38:09.887399 25920 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:38:09.890015 26122 tablet_service.cc:1467] Processing CreateTablet for tablet bee663da416245a0a2b2a719aa209288 (DEFAULT_TABLE table=test-table [id=0d2d93c1ac694e64b956fccd6566f7e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:38:09.890259 26122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bee663da416245a0a2b2a719aa209288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:09.892005 26052 tablet_bootstrap.cc:492] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d: Bootstrap starting.
I20260811 14:38:09.892360 26052 tablet_bootstrap.cc:654] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:09.892957 26052 tablet_bootstrap.cc:492] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d: No bootstrap required, opened a new log
I20260811 14:38:09.893018 26052 ts_tablet_manager.cc:1397] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260811 14:38:09.893124 26052 raft_consensus.cc:348] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } }
I20260811 14:38:09.893203 26052 raft_consensus.cc:374] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:09.893257 26052 raft_consensus.cc:729] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56f4d08c06f146a89480fa9e5a591c2d, State: Initialized, Role: FOLLOWER
I20260811 14:38:09.893311 26052 consensus_queue.cc:260] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } }
I20260811 14:38:09.893410 26052 ts_tablet_manager.cc:1428] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:09.894347 26242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.889435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:46803 (local address 127.0.0.1:48552)
0811 14:38:09.889505 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:09.889519 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:09.889566 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:09.890030 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:09.890031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:09.890032 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:09.890076 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.890078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.891236 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.891238 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.892043 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.892045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.894092 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.894093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.894159 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.894164 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:38:09.894221 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 14:38:09.894270 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 14:38:09.894412 26724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.889552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48552 (local address 127.2.103.66:46803)
0811 14:38:09.889849 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:09.889854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:09.889865 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:38:09.889910 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:09.889912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:09.889925 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:09.890019 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:09.890164 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.891124 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.892907 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.894018 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.894096 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.894168 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:09.894289 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 14:38:09.894335 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":125,"threads_started":1}
I20260811 14:38:09.894920 26722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.889539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:37569 (local address 127.0.0.1:42086)
0811 14:38:09.889651 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:09.889659 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:09.889699 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:09.890089 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:09.890090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:09.890092 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:09.890138 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.890140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.891826 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.891827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.893598 (+ 1771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.893601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.894668 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.894669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.894739 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.894744 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:38:09.894798 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 14:38:09.894842 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260811 14:38:09.895143 26725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.889640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42086 (local address 127.2.103.67:37569)
0811 14:38:09.889850 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:09.889855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:09.889865 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:38:09.889907 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:09.889909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:09.889926 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:09.890009 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:09.890220 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.891709 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.893683 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.894420 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.894499 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.894571 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:09.895025 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 14:38:09.895070 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":119,"threads_started":1}
I20260811 14:38:09.895270 26357 tablet_service.cc:1467] Processing CreateTablet for tablet bee663da416245a0a2b2a719aa209288 (DEFAULT_TABLE table=test-table [id=0d2d93c1ac694e64b956fccd6566f7e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:38:09.895610 26357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bee663da416245a0a2b2a719aa209288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:09.898394 26728 tablet_bootstrap.cc:492] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551: Bootstrap starting.
I20260811 14:38:09.899137 26728 tablet_bootstrap.cc:654] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:09.899135 26608 tablet_service.cc:1467] Processing CreateTablet for tablet bee663da416245a0a2b2a719aa209288 (DEFAULT_TABLE table=test-table [id=0d2d93c1ac694e64b956fccd6566f7e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:38:09.899549 26608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bee663da416245a0a2b2a719aa209288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:09.899780 26728 log.cc:824] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:09.900650 26728 tablet_bootstrap.cc:492] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551: No bootstrap required, opened a new log
I20260811 14:38:09.900727 26728 ts_tablet_manager.cc:1397] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:38:09.902824 26728 raft_consensus.cc:348] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } }
I20260811 14:38:09.903024 26728 raft_consensus.cc:374] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:09.903062 26728 raft_consensus.cc:729] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba71ee3655164ecba3f4063134037551, State: Initialized, Role: FOLLOWER
I20260811 14:38:09.903196 26728 consensus_queue.cc:260] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } }
I20260811 14:38:09.903350 26731 tablet_bootstrap.cc:492] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e: Bootstrap starting.
I20260811 14:38:09.903494 26493 heartbeater.cc:503] Master 127.2.103.126:46077 was elected leader, sending a full tablet report...
I20260811 14:38:09.903750 26728 ts_tablet_manager.cc:1428] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:38:09.904088 26731 tablet_bootstrap.cc:654] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:09.904743 26731 log.cc:824] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:09.905607 26731 tablet_bootstrap.cc:492] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e: No bootstrap required, opened a new log
I20260811 14:38:09.905709 26731 ts_tablet_manager.cc:1397] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:38:09.907220 26731 raft_consensus.cc:348] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } }
I20260811 14:38:09.907424 26731 raft_consensus.cc:374] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:09.907474 26731 raft_consensus.cc:729] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6aad9e62431c49b9b115d08925eae94e, State: Initialized, Role: FOLLOWER
I20260811 14:38:09.907600 26731 consensus_queue.cc:260] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } }
I20260811 14:38:09.907893 26714 heartbeater.cc:503] Master 127.2.103.126:46077 was elected leader, sending a full tablet report...
I20260811 14:38:09.907909 26731 ts_tablet_manager.cc:1428] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 14:38:09.978926 26241 raft_consensus.cc:482] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:38:09.979118 26241 raft_consensus.cc:504] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } }
I20260811 14:38:09.979576 26241 leader_election.cc:288] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba71ee3655164ecba3f4063134037551 (127.2.103.66:46803), 6aad9e62431c49b9b115d08925eae94e (127.2.103.67:37569)
I20260811 14:38:09.983141 26724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.979914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:38651 (local address 127.2.103.66:46803)
0811 14:38:09.979966 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:09.979969 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:09.979980 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:38:09.980043 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:09.980044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:09.980054 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:09.980118 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:09.980313 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.981044 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.982168 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.982789 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.982845 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.982853 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:09.983048 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 14:38:09.983083 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 14:38:09.983176 26235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.979789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:46803 (local address 127.2.103.65:38651)
0811 14:38:09.979867 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:09.979889 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:09.979981 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:09.980143 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:09.980144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:09.980146 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:09.980211 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.980213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.981130 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.981131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.982070 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.982072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.982891 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.982892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.982970 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.982974 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:38:09.983033 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 14:38:09.983078 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 14:38:09.983619 26738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.979925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:37569 (local address 127.2.103.65:50841)
0811 14:38:09.980113 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:09.980123 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:09.980173 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:09.980322 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:09.980323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:09.980324 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:09.980752 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.980755 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.981628 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.981629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.982570 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:09.982573 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.983369 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.983370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:09.983437 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.983440 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:38:09.983490 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 14:38:09.983526 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":107,"threads_started":1}
I20260811 14:38:09.983625 26725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:09.980023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:50841 (local address 127.2.103.67:37569)
0811 14:38:09.980072 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:09.980074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:09.980165 (+ 91us) server_negotiation.cc:407] Connection header received
0811 14:38:09.980239 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:09.980240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:09.980251 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:09.980304 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:09.980839 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.981541 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.982701 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:09.983251 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:09.983327 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:09.983337 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:09.983515 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 14:38:09.983563 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 14:38:09.983561 26412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bee663da416245a0a2b2a719aa209288" candidate_uuid: "56f4d08c06f146a89480fa9e5a591c2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba71ee3655164ecba3f4063134037551" is_pre_election: true
I20260811 14:38:09.983742 26412 raft_consensus.cc:2454] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56f4d08c06f146a89480fa9e5a591c2d in term 0.
I20260811 14:38:09.983964 26044 leader_election.cc:302] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56f4d08c06f146a89480fa9e5a591c2d, ba71ee3655164ecba3f4063134037551; no voters:
I20260811 14:38:09.983975 26641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bee663da416245a0a2b2a719aa209288" candidate_uuid: "56f4d08c06f146a89480fa9e5a591c2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6aad9e62431c49b9b115d08925eae94e" is_pre_election: true
I20260811 14:38:09.984123 26641 raft_consensus.cc:2454] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56f4d08c06f146a89480fa9e5a591c2d in term 0.
I20260811 14:38:09.984110 26241 raft_consensus.cc:2781] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:38:09.984184 26241 raft_consensus.cc:482] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:38:09.984218 26241 raft_consensus.cc:3037] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:38:09.985003 26241 raft_consensus.cc:504] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } }
I20260811 14:38:09.985119 26241 leader_election.cc:288] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 1 election: Requested vote from peers ba71ee3655164ecba3f4063134037551 (127.2.103.66:46803), 6aad9e62431c49b9b115d08925eae94e (127.2.103.67:37569)
I20260811 14:38:09.985280 26412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bee663da416245a0a2b2a719aa209288" candidate_uuid: "56f4d08c06f146a89480fa9e5a591c2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba71ee3655164ecba3f4063134037551"
I20260811 14:38:09.985394 26412 raft_consensus.cc:3037] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:38:09.985380 26641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bee663da416245a0a2b2a719aa209288" candidate_uuid: "56f4d08c06f146a89480fa9e5a591c2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6aad9e62431c49b9b115d08925eae94e"
I20260811 14:38:09.985491 26641 raft_consensus.cc:3037] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e [term 0 FOLLOWER]: Advancing to term 1
W20260811 14:38:09.985841 26494 tablet.cc:2369] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:38:09.986434 26641 raft_consensus.cc:2454] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56f4d08c06f146a89480fa9e5a591c2d in term 1.
I20260811 14:38:09.986447 26412 raft_consensus.cc:2454] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56f4d08c06f146a89480fa9e5a591c2d in term 1.
I20260811 14:38:09.986640 26044 leader_election.cc:302] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56f4d08c06f146a89480fa9e5a591c2d, ba71ee3655164ecba3f4063134037551; no voters:
I20260811 14:38:09.986786 26241 raft_consensus.cc:2781] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:38:09.986940 26241 raft_consensus.cc:686] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 1 LEADER]: Becoming Leader. State: Replica: 56f4d08c06f146a89480fa9e5a591c2d, State: Running, Role: LEADER
I20260811 14:38:09.987001 26241 consensus_queue.cc:237] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } }
I20260811 14:38:09.987901 25920 catalog_manager.cc:5495] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 56f4d08c06f146a89480fa9e5a591c2d (127.2.103.65). New cstate: current_term: 1 leader_uuid: "56f4d08c06f146a89480fa9e5a591c2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } health_report { overall_health: HEALTHY } } }
I20260811 14:38:10.009333 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:10.006996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54446 (local address 127.2.103.65:35451)
0811 14:38:10.007041 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:10.007044 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:10.007060 (+ 16us) server_negotiation.cc:407] Connection header received
0811 14:38:10.007137 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:10.007138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:10.007157 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:10.007227 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:38:10.007375 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:10.007836 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:10.008618 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:10.008805 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:10.008858 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:10.009011 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:38:10.009117 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:38:10.009174 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:10.009252 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 14:38:10.009271 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 14:38:10.011085 26641 raft_consensus.cc:1261] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e [term 1 FOLLOWER]: Refusing update from remote peer 56f4d08c06f146a89480fa9e5a591c2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:38:10.011092 26412 raft_consensus.cc:1261] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551 [term 1 FOLLOWER]: Refusing update from remote peer 56f4d08c06f146a89480fa9e5a591c2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:38:10.011505 26739 consensus_queue.cc:1035] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:38:10.011693 26241 consensus_queue.cc:1035] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:38:10.016759 26752 mvcc.cc:205] Tried to move back new op lower bound from 7317336432682295296 to 7317336432587317248. Current Snapshot: MvccSnapshot[applied={T|T < 7317336432682295296}]
I20260811 14:38:10.016764 25920 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:58192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:38:10.016871 25920 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:58192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:38:10.017621 25920 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb: Sending DeleteTablet for 3 replicas of tablet bee663da416245a0a2b2a719aa209288
I20260811 14:38:10.017978 26357 tablet_service.cc:1514] Processing DeleteTablet for tablet bee663da416245a0a2b2a719aa209288 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:38:10 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:48552
I20260811 14:38:10.018018 26608 tablet_service.cc:1514] Processing DeleteTablet for tablet bee663da416245a0a2b2a719aa209288 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:38:10 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:42086
I20260811 14:38:10.018111 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 25987
I20260811 14:38:10.020960 26769 tablet_replica.cc:330] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e: stopping tablet replica
I20260811 14:38:10.021130 26769 raft_consensus.cc:2229] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:38:10.021288 26769 raft_consensus.cc:2258] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:38:10.024931 26769 ts_tablet_manager.cc:1905] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:38:10.026829 26769 ts_tablet_manager.cc:1918] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:38:10.026937 26769 log.cc:1196] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/bee663da416245a0a2b2a719aa209288
I20260811 14:38:10.027207 26769 ts_tablet_manager.cc:1939] T bee663da416245a0a2b2a719aa209288 P 6aad9e62431c49b9b115d08925eae94e: Deleting consensus metadata
I20260811 14:38:10.027781 25895 catalog_manager.cc:4841] TS 6aad9e62431c49b9b115d08925eae94e (127.2.103.67:37569): tablet bee663da416245a0a2b2a719aa209288 (table test-table [id=0d2d93c1ac694e64b956fccd6566f7e6]) successfully deleted
I20260811 14:38:10.031755 26763 tablet_replica.cc:330] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551: stopping tablet replica
I20260811 14:38:10.031932 26763 raft_consensus.cc:2229] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:38:10.032074 26763 raft_consensus.cc:2258] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:38:10.032652 26763 ts_tablet_manager.cc:1905] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:38:10.034184 26763 ts_tablet_manager.cc:1918] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:38:10.034289 26763 log.cc:1196] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/bee663da416245a0a2b2a719aa209288
I20260811 14:38:10.034698 26763 ts_tablet_manager.cc:1939] T bee663da416245a0a2b2a719aa209288 P ba71ee3655164ecba3f4063134037551: Deleting consensus metadata
I20260811 14:38:10.035233 25895 catalog_manager.cc:4841] TS ba71ee3655164ecba3f4063134037551 (127.2.103.66:46803): tablet bee663da416245a0a2b2a719aa209288 (table test-table [id=0d2d93c1ac694e64b956fccd6566f7e6]) successfully deleted
W20260811 14:38:10.037051 25895 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.103.65:35451 (error 108)
I20260811 14:38:10.037539 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 26240
I20260811 14:38:10.037714 26722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:10.037475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:35451 (local address 127.0.0.1:54448)
0811 14:38:10.037588 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:10.037669 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.65:35451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260811 14:38:10.051414 25895 catalog_manager.cc:4584] TS 56f4d08c06f146a89480fa9e5a591c2d (127.2.103.65:35451): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bee663da416245a0a2b2a719aa209288: Network error: Client connection negotiation failed: client connection to 127.2.103.65:35451: connect: Connection refused (error 111)
I20260811 14:38:10.069736 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 26498
I20260811 14:38:10.087509 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 25863
I20260811 14:38:10.103459 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:46077
--webserver_interface=127.2.103.126
--webserver_port=32981
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:46077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:38:10.190814 26772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:10.191082 26772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:10.191108 26772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:10.192337 26772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:38:10.192389 26772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:10.192404 26772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:10.192416 26772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:38:10.192430 26772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:38:10.194087 26772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:46077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:46077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=32981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:10.194334 26772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:10.194531 26772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:10.200215 26782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:10.200242 26780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:10.200242 26772 server_base.cc:1034] running on GCE node
W20260811 14:38:10.200219 26785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:10.200788 26772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:10.201076 26772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:10.202257 26772 hybrid_clock.cc:648] HybridClock initialized: now 1786459090202228 us; error 45 us; skew 500 ppm
Aug 11 14:38:10 dist-test-slave-twwt krb5kdc[24963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459090, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:10.205259 26772 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:38:10.205811 26772 webserver.cc:460] Webserver started at http://127.2.103.126:32981/ using document root <none> and password file <none>
I20260811 14:38:10.206085 26772 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:10.206159 26772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:10.207721 26772 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:10.208595 26797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:10.208845 26772 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:10.208969 26772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "15a67e27de6440389dc1877bcec288bb"
format_stamp: "Formatted at 2026-08-11 14:38:08 on dist-test-slave-twwt"
server_key: "fc713927050ed3c69d37291d769b23c4"
server_key_iv: "70a0640f68f641bf336fb52135d056fc"
server_key_version: "encryptionkey@0"
I20260811 14:38:10.209430 26772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:10.229895 26772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:10.232576 26772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:10.232785 26772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:10.237612 26772 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:46077
I20260811 14:38:10.237651 26893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:46077 every 8 connection(s)
I20260811 14:38:10.238087 26772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:38:10.239686 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 26772
I20260811 14:38:10.239969 26896 sys_catalog.cc:263] Verifying existing consensus state
I20260811 14:38:10.240023 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:35451
--local_ip_for_outbound_sockets=127.2.103.65
--tserver_master_addrs=127.2.103.126:46077
--webserver_port=46355
--webserver_interface=127.2.103.65
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:38:10.240720 26896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb: Bootstrap starting.
I20260811 14:38:10.249791 26896 log.cc:824] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:10.252228 26896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:38:10.252568 26896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb: Bootstrap complete.
I20260811 14:38:10.255038 26896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a67e27de6440389dc1877bcec288bb" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46077 } }
I20260811 14:38:10.255300 26896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15a67e27de6440389dc1877bcec288bb, State: Initialized, Role: FOLLOWER
I20260811 14:38:10.255493 26896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a67e27de6440389dc1877bcec288bb" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46077 } }
I20260811 14:38:10.255575 26896 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:38:10.255630 26896 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:38:10.255684 26896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 2 FOLLOWER]: Advancing to term 3
I20260811 14:38:10.256734 26896 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a67e27de6440389dc1877bcec288bb" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46077 } }
I20260811 14:38:10.256871 26896 leader_election.cc:302] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15a67e27de6440389dc1877bcec288bb; no voters:
I20260811 14:38:10.257092 26896 leader_election.cc:288] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 14:38:10.257134 26905 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 3 FOLLOWER]: Leader election won for term 3
I20260811 14:38:10.257254 26905 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [term 3 LEADER]: Becoming Leader. State: Replica: 15a67e27de6440389dc1877bcec288bb, State: Running, Role: LEADER
I20260811 14:38:10.257392 26896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:38:10.257364 26905 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a67e27de6440389dc1877bcec288bb" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46077 } }
I20260811 14:38:10.257660 26905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15a67e27de6440389dc1877bcec288bb. Latest consensus state: current_term: 3 leader_uuid: "15a67e27de6440389dc1877bcec288bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a67e27de6440389dc1877bcec288bb" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46077 } } }
I20260811 14:38:10.257740 26905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [sys.catalog]: This master's current role is: LEADER
I20260811 14:38:10.257666 26906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "15a67e27de6440389dc1877bcec288bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a67e27de6440389dc1877bcec288bb" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46077 } } }
I20260811 14:38:10.257913 26906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15a67e27de6440389dc1877bcec288bb [sys.catalog]: This master's current role is: LEADER
I20260811 14:38:10.257970 26910 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:38:10.258878 26910 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:38:10.259006 26910 catalog_manager.cc:1259] Loaded cluster ID: 85d8e8292529415ab265ea17ab5acacd
I20260811 14:38:10.259049 26910 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 14:38:10.259456 26910 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6e7b7de9ad google::LogMessage::Fail() at ??:0
@ 0x7f6e7b7e188a google::LogMessage::SendToLog() at ??:0
@ 0x7f6e7b7de4de google::LogMessage::Flush() at ??:0
@ 0x7f6e7b7dfa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6e836ff0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f6e837088b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6e7d6ac0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6e7d6a1954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6e7e5056db start_thread at ??:0
@ 0x7f6e7a2dc61f clone at ??:0
W20260811 14:38:10.340957 26899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:10.341178 26899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:10.341204 26899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:10.342427 26899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:10.342479 26899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:10.342507 26899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:38:10.344146 26899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:35451
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=46355
--tserver_master_addrs=127.2.103.126:46077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:10.344434 26899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:10.344718 26899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:10.350436 26955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:10.350585 26899 server_base.cc:1034] running on GCE node
W20260811 14:38:10.350587 26958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:10.350436 26956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:10.351032 26899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:10.351359 26899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:10.352553 26899 hybrid_clock.cc:648] HybridClock initialized: now 1786459090352539 us; error 29 us; skew 500 ppm
Aug 11 14:38:10 dist-test-slave-twwt krb5kdc[24963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459090, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:10.355484 26899 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:38:10.356076 26899 webserver.cc:460] Webserver started at http://127.2.103.65:46355/ using document root <none> and password file <none>
I20260811 14:38:10.356325 26899 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:10.356400 26899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:10.357954 26899 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 14:38:10.358915 26973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:10.359189 26899 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:10.359336 26899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "56f4d08c06f146a89480fa9e5a591c2d"
format_stamp: "Formatted at 2026-08-11 14:38:08 on dist-test-slave-twwt"
server_key: "3cf50b218126343ca6835b982cac5fc5"
server_key_iv: "f332a75a58b7a499c09cd08c1af57ee2"
server_key_version: "encryptionkey@0"
I20260811 14:38:10.359745 26899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:10.371728 26899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:10.374750 26899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:10.374953 26899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:10.375253 26899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:38:10.375761 26989 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 14:38:10.378103 26899 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 14:38:10.378178 26899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260811 14:38:10.378227 26899 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 14:38:10.379050 26899 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 14:38:10.379113 26899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260811 14:38:10.379202 26989 tablet_bootstrap.cc:492] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d: Bootstrap starting.
I20260811 14:38:10.386605 26899 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:35451
I20260811 14:38:10.386955 27167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:35451 every 8 connection(s)
I20260811 14:38:10.387194 26899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:38:10.392270 26989 log.cc:824] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:10.395601 26989 tablet_bootstrap.cc:492] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:38:10.396057 26989 tablet_bootstrap.cc:492] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d: Bootstrap complete.
I20260811 14:38:10.396689 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 26899
I20260811 14:38:10.396960 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:46803
--local_ip_for_outbound_sockets=127.2.103.66
--tserver_master_addrs=127.2.103.126:46077
--webserver_port=40787
--webserver_interface=127.2.103.66
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:38:10.397524 26989 ts_tablet_manager.cc:1397] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.010s
I20260811 14:38:10.399262 26989 raft_consensus.cc:348] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } }
I20260811 14:38:10.399719 26989 raft_consensus.cc:729] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56f4d08c06f146a89480fa9e5a591c2d, State: Initialized, Role: FOLLOWER
I20260811 14:38:10.399988 26989 consensus_queue.cc:260] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } }
I20260811 14:38:10.400385 26989 ts_tablet_manager.cc:1428] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d: Time spent starting tablet: real 0.003s user 0.006s sys 0.000s
W20260811 14:38:10.482789 27176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:10.483120 27176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:10.483162 27176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:10.484428 27176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:10.484506 27176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:10.484555 27176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:38:10.486159 27176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:46803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=40787
--tserver_master_addrs=127.2.103.126:46077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:10.486480 27176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:10.486824 27176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:10.492245 27191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:10.492245 27186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:10.492609 27176 server_base.cc:1034] running on GCE node
W20260811 14:38:10.492664 27187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:10.492939 27176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:10.493242 27176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:10.494446 27176 hybrid_clock.cc:648] HybridClock initialized: now 1786459090494427 us; error 29 us; skew 500 ppm
Aug 11 14:38:10 dist-test-slave-twwt krb5kdc[24963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459090, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:10.497995 27176 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:38:10.498574 27176 webserver.cc:460] Webserver started at http://127.2.103.66:40787/ using document root <none> and password file <none>
I20260811 14:38:10.498910 27176 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:10.498994 27176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:10.510799 27176 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.002s sys 0.000s
I20260811 14:38:10.511638 27204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:10.511898 27176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:10.512022 27176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "ba71ee3655164ecba3f4063134037551"
format_stamp: "Formatted at 2026-08-11 14:38:08 on dist-test-slave-twwt"
server_key: "9e75505cb97a1336f6268c97cc5ebbce"
server_key_iv: "ed753235864df63dd4ce5e6519f83353"
server_key_version: "encryptionkey@0"
I20260811 14:38:10.512465 27176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260811 14:38:10.515693 27170 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:38:10.387890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.65:46037)
0811 14:38:10.388072 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:10.388118 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:10.388644 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:10.515470 (+126826us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":103,"threads_started":1}
W20260811 14:38:10.515929 27168 heartbeater.cc:644] Failed to heartbeat to 127.2.103.126:46077 (0 consecutive failures): Network error: Failed to ping master at 127.2.103.126:46077: Client connection negotiation failed: client connection to 127.2.103.126:46077: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 14:38:10.516330 27170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:10.516183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.65:45761)
0811 14:38:10.516247 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:10.516302 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 14:38:10.516698 27170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:10.516590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.65:46399)
0811 14:38:10.516639 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:10.516671 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 14:38:10.516863 27168 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 14:38:10.526397 27176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:10.528481 27176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:10.528642 27176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:10.528930 27176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:38:10.529259 27176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:38:10.529294 27176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:10.529338 27176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:38:10.529366 27176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:10.535629 27176 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:46803
I20260811 14:38:10.535681 27383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:46803 every 8 connection(s)
I20260811 14:38:10.536044 27176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:38:10.537132 27387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:10.536750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.66:49671)
0811 14:38:10.536918 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:10.536992 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":76,"threads_started":1}
W20260811 14:38:10.537551 27384 heartbeater.cc:644] Failed to heartbeat to 127.2.103.126:46077 (0 consecutive failures): Network error: Failed to ping master at 127.2.103.126:46077: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
I20260811 14:38:10.537827 27387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:10.537733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.66:42289)
0811 14:38:10.537782 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:10.537809 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 14:38:10.538110 27387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:10.538037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.66:55697)
0811 14:38:10.538075 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:10.538096 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260811 14:38:10.538228 27384 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 14:38:10.539069 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 27176
I20260811 14:38:10.539356 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:37569
--local_ip_for_outbound_sockets=127.2.103.67
--tserver_master_addrs=127.2.103.126:46077
--webserver_port=42505
--webserver_interface=127.2.103.67
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:38:10.626886 27388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:10.627112 27388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:10.627138 27388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:10.628266 27388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:10.628302 27388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:10.628329 27388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:38:10.629745 27388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:37569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=42505
--tserver_master_addrs=127.2.103.126:46077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:10.629940 27388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:10.630142 27388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:10.635447 27400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:10.635453 27395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:10.635708 27388 server_base.cc:1034] running on GCE node
W20260811 14:38:10.635562 27398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:10.636029 27388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:10.636296 27388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:10.637481 27388 hybrid_clock.cc:648] HybridClock initialized: now 1786459090637467 us; error 31 us; skew 500 ppm
Aug 11 14:38:10 dist-test-slave-twwt krb5kdc[24963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459090, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:10.640192 27388 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:38:10.640661 27388 webserver.cc:460] Webserver started at http://127.2.103.67:42505/ using document root <none> and password file <none>
I20260811 14:38:10.640944 27388 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:10.641022 27388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:10.642453 27388 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:38:10.643137 27410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:10.643318 27388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:38:10.643435 27388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "6aad9e62431c49b9b115d08925eae94e"
format_stamp: "Formatted at 2026-08-11 14:38:09 on dist-test-slave-twwt"
server_key: "0c31573b8791ff1c85d97464d4420974"
server_key_iv: "fb748f6ab2a48565586a6ef29eeaad44"
server_key_version: "encryptionkey@0"
I20260811 14:38:10.643812 27388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:10.662338 27388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:10.664570 27388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:10.664747 27388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:10.665023 27388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:38:10.665372 27388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:38:10.665428 27388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:10.665472 27388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:38:10.665498 27388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:10.671909 27388 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:37569
I20260811 14:38:10.671949 27579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:37569 every 8 connection(s)
I20260811 14:38:10.672329 27388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:38:10.673452 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:10.673046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.67:57123)
0811 14:38:10.673222 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:10.673299 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
W20260811 14:38:10.673871 27580 heartbeater.cc:644] Failed to heartbeat to 127.2.103.126:46077 (0 consecutive failures): Network error: Failed to ping master at 127.2.103.126:46077: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
I20260811 14:38:10.674161 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:10.674066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.67:41827)
0811 14:38:10.674116 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:10.674144 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 14:38:10.674410 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:10.674361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.67:57247)
0811 14:38:10.674382 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:10.674399 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260811 14:38:10.674520 27580 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 14:38:10.675990 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 27388
I20260811 14:38:11.517570 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:11.517295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.65:38717)
0811 14:38:11.517458 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:11.517502 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":75,"threads_started":1}
I20260811 14:38:11.538829 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:11.538576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.66:57861)
0811 14:38:11.538744 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:11.538779 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":104,"threads_started":1}
I20260811 14:38:11.675159 27587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:11.674901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.67:58545)
0811 14:38:11.675053 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:11.675099 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":64,"threads_started":1}
I20260811 14:38:11.861248 27588 raft_consensus.cc:482] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:38:11.861441 27588 raft_consensus.cc:504] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } }
I20260811 14:38:11.861832 27588 leader_election.cc:288] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba71ee3655164ecba3f4063134037551 (127.2.103.66:46803), 6aad9e62431c49b9b115d08925eae94e (127.2.103.67:37569)
Aug 11 14:38:11 dist-test-slave-twwt krb5kdc[24963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459090, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.67@KRBTEST.COM
Aug 11 14:38:11 dist-test-slave-twwt krb5kdc[24963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459090, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.66@KRBTEST.COM
I20260811 14:38:11.875653 27592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:11.862527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:57653 (local address 127.2.103.66:46803)
0811 14:38:11.862697 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:11.862710 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:11.862750 (+ 40us) server_negotiation.cc:407] Connection header received
0811 14:38:11.863431 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:11.863435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:11.863509 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:11.863677 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:11.864687 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:11.866124 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:11.867679 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:11.868338 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:11.868445 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:11.871763 (+ 3318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:11.871767 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:11.871767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:11.871777 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:11.873626 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:11.874095 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:11.874098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:11.874098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:11.874228 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:11.874567 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:11.874568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:11.874568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:11.874929 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:11.875006 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:11.875392 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 14:38:11.875535 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260811 14:38:11.875654 27591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:11.862402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:38261 (local address 127.2.103.67:37569)
0811 14:38:11.862898 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:11.862905 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:11.862933 (+ 28us) server_negotiation.cc:407] Connection header received
0811 14:38:11.863456 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:11.863458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:11.863508 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:11.863677 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:11.864687 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:11.866125 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:11.867680 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:11.868322 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:11.868445 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:11.871375 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:11.871379 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:11.871379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:11.871390 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:11.873522 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:11.874105 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:11.874108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:11.874108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:11.874228 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:11.874578 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:11.874579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:11.874579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:11.874913 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:11.875006 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:11.875392 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 14:38:11.875534 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":83,"threads_started":1}
I20260811 14:38:11.875661 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:11.862186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:37569 (local address 127.2.103.65:38261)
0811 14:38:11.862611 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:11.862649 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:11.862703 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:11.863747 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:11.863750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:11.864267 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:11.864541 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:11.864551 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:11.866261 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:11.866261 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:11.867584 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:11.867587 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:11.868437 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:11.868437 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:11.868586 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:11.869275 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:11.869287 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:11.871241 (+ 1954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:11.873605 (+ 2364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:11.873611 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:11.873613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:11.874006 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:11.874304 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:11.874305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:11.874305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:11.874492 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:11.875001 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:11.875009 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:11.875177 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 14:38:11.875316 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 14:38:11.875535 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"mutex_wait_us":84,"thread_start_us":49,"threads_started":1}
I20260811 14:38:11.875710 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:11.862060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:46803 (local address 127.2.103.65:57653)
0811 14:38:11.862121 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:11.862134 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:11.862213 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:11.863747 (+ 1534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:11.863750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:11.864267 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:11.864541 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:11.864551 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:11.866248 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:11.866249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:11.867584 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:11.867587 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:11.868434 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:11.868436 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:11.868586 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:11.869285 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:11.869289 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:11.871651 (+ 2362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:11.873714 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:11.873715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:11.873715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:11.874007 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:11.874304 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:11.874305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:11.874305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:11.874492 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:11.875001 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:11.875009 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:11.875177 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 14:38:11.875311 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 14:38:11.875592 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":191}
W20260811 14:38:11.876818 26978 leader_election.cc:341] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6aad9e62431c49b9b115d08925eae94e (127.2.103.67:37569): Not found: Tablet not found: bee663da416245a0a2b2a719aa209288
W20260811 14:38:11.877326 26978 leader_election.cc:341] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ba71ee3655164ecba3f4063134037551 (127.2.103.66:46803): Not found: Tablet not found: bee663da416245a0a2b2a719aa209288
I20260811 14:38:11.877393 26978 leader_election.cc:302] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56f4d08c06f146a89480fa9e5a591c2d; no voters: 6aad9e62431c49b9b115d08925eae94e, ba71ee3655164ecba3f4063134037551
I20260811 14:38:11.877580 27588 raft_consensus.cc:2726] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 14:38:12.518414 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:12.518159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.65:50213)
0811 14:38:12.518316 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:12.518365 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260811 14:38:12.539568 27595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:12.539305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.66:54645)
0811 14:38:12.539476 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:12.539522 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":73,"threads_started":1}
I20260811 14:38:12.675907 27597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:12.675670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.67:37931)
0811 14:38:12.675807 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:12.675849 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260811 14:38:13.519276 27598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:13.519035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.65:57463)
0811 14:38:13.519181 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:13.519228 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":66,"threads_started":1}
I20260811 14:38:13.540395 27599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:13.540158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.66:38053)
0811 14:38:13.540314 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:13.540353 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
I20260811 14:38:13.560438 27600 raft_consensus.cc:482] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:38:13.560528 27600 raft_consensus.cc:504] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } }
I20260811 14:38:13.560726 27600 leader_election.cc:288] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba71ee3655164ecba3f4063134037551 (127.2.103.66:46803), 6aad9e62431c49b9b115d08925eae94e (127.2.103.67:37569)
W20260811 14:38:13.561261 26978 leader_election.cc:341] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6aad9e62431c49b9b115d08925eae94e (127.2.103.67:37569): Not found: Tablet not found: bee663da416245a0a2b2a719aa209288
W20260811 14:38:13.561357 26978 leader_election.cc:341] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ba71ee3655164ecba3f4063134037551 (127.2.103.66:46803): Not found: Tablet not found: bee663da416245a0a2b2a719aa209288
I20260811 14:38:13.561386 26978 leader_election.cc:302] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56f4d08c06f146a89480fa9e5a591c2d; no voters: 6aad9e62431c49b9b115d08925eae94e, ba71ee3655164ecba3f4063134037551
I20260811 14:38:13.561477 27600 raft_consensus.cc:2726] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 14:38:13.676622 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:13.676393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.67:57483)
0811 14:38:13.676539 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:13.676577 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":84,"threads_started":1}
I20260811 14:38:14.520087 27603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:14.519843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.65:55233)
0811 14:38:14.519985 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:14.520027 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260811 14:38:14.541065 27604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:14.540870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.66:36087)
0811 14:38:14.540991 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:14.541021 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
I20260811 14:38:14.677446 27609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:14.677175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.67:35137)
0811 14:38:14.677352 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:14.677397 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":73,"threads_started":1}
I20260811 14:38:15.520892 27610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:15.520622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.65:49227)
0811 14:38:15.520764 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:15.520842 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":68,"threads_started":1}
I20260811 14:38:15.541728 27611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:15.541496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.66:55143)
0811 14:38:15.541651 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:15.541682 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":102,"threads_started":1}
I20260811 14:38:15.678352 27613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:15.678098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.67:47643)
0811 14:38:15.678254 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:15.678287 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260811 14:38:15.991804 27614 raft_consensus.cc:482] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:38:15.991911 27614 raft_consensus.cc:504] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } }
I20260811 14:38:15.992074 27614 leader_election.cc:288] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba71ee3655164ecba3f4063134037551 (127.2.103.66:46803), 6aad9e62431c49b9b115d08925eae94e (127.2.103.67:37569)
W20260811 14:38:15.992545 26978 leader_election.cc:341] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ba71ee3655164ecba3f4063134037551 (127.2.103.66:46803): Not found: Tablet not found: bee663da416245a0a2b2a719aa209288
W20260811 14:38:15.992635 26978 leader_election.cc:341] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6aad9e62431c49b9b115d08925eae94e (127.2.103.67:37569): Not found: Tablet not found: bee663da416245a0a2b2a719aa209288
I20260811 14:38:15.992690 26978 leader_election.cc:302] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56f4d08c06f146a89480fa9e5a591c2d; no voters: 6aad9e62431c49b9b115d08925eae94e, ba71ee3655164ecba3f4063134037551
I20260811 14:38:15.992827 27614 raft_consensus.cc:2726] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 14:38:16.521739 27615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:16.521462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.65:34389)
0811 14:38:16.521637 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:16.521686 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":90,"threads_started":1}
I20260811 14:38:16.542440 27616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:16.542193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.66:45879)
0811 14:38:16.542359 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:16.542395 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":111,"threads_started":1}
I20260811 14:38:16.679040 27617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:16.678825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.67:45445)
0811 14:38:16.678958 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:16.678996 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260811 14:38:17.522554 27618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:17.522306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.65:51069)
0811 14:38:17.522458 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:17.522506 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":66,"threads_started":1}
I20260811 14:38:17.543208 27619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:17.542970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.66:40139)
0811 14:38:17.543122 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:17.543163 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":66,"threads_started":1}
I20260811 14:38:17.679890 27621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:17.679625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.67:45781)
0811 14:38:17.679791 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:17.679837 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":66,"threads_started":1}
I20260811 14:38:18.494381 27622 raft_consensus.cc:482] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:38:18.494510 27622 raft_consensus.cc:504] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } }
I20260811 14:38:18.494737 27622 leader_election.cc:288] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba71ee3655164ecba3f4063134037551 (127.2.103.66:46803), 6aad9e62431c49b9b115d08925eae94e (127.2.103.67:37569)
W20260811 14:38:18.495224 26978 leader_election.cc:341] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ba71ee3655164ecba3f4063134037551 (127.2.103.66:46803): Not found: Tablet not found: bee663da416245a0a2b2a719aa209288
W20260811 14:38:18.495327 26978 leader_election.cc:341] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6aad9e62431c49b9b115d08925eae94e (127.2.103.67:37569): Not found: Tablet not found: bee663da416245a0a2b2a719aa209288
I20260811 14:38:18.495380 26978 leader_election.cc:302] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56f4d08c06f146a89480fa9e5a591c2d; no voters: 6aad9e62431c49b9b115d08925eae94e, ba71ee3655164ecba3f4063134037551
I20260811 14:38:18.495499 27622 raft_consensus.cc:2726] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 14:38:18.523344 27624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:18.523106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.65:37503)
0811 14:38:18.523255 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:18.523294 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":71,"threads_started":1}
I20260811 14:38:18.543990 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:18.543754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.66:60471)
0811 14:38:18.543900 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:18.543940 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260811 14:38:18.680661 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:18.680412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.67:53771)
0811 14:38:18.680559 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:18.680595 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
I20260811 14:38:19.524142 27630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:19.523868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.65:51235)
0811 14:38:19.524039 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:19.524089 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":84,"threads_started":1}
I20260811 14:38:19.544678 27631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:19.544445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.66:40729)
0811 14:38:19.544598 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:19.544631 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260811 14:38:19.681491 27632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:19.681231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.67:35843)
0811 14:38:19.681368 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:19.681431 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":63,"threads_started":1}
I20260811 14:38:20.525079 27633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:20.524809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.65:35175)
0811 14:38:20.524980 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:20.525029 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":77,"threads_started":1}
I20260811 14:38:20.545487 27635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:20.545240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.66:41953)
0811 14:38:20.545393 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:20.545439 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":68,"threads_started":1}
I20260811 14:38:20.682292 27636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:20.682025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.67:53829)
0811 14:38:20.682188 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:20.682237 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":79,"threads_started":1}
I20260811 14:38:21.525923 27639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:21.525618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.65:49989)
0811 14:38:21.525832 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:21.525873 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":107,"threads_started":1}
I20260811 14:38:21.546279 27648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:21.546048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.66:57109)
0811 14:38:21.546182 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:21.546229 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
I20260811 14:38:21.683142 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:21.682867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.67:52357)
0811 14:38:21.683042 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:21.683090 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":107,"threads_started":1}
I20260811 14:38:22.526749 27650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:22.526425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.65:48521)
0811 14:38:22.526588 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:22.526699 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":67,"threads_started":1}
I20260811 14:38:22.547063 27651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:22.546835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.66:32817)
0811 14:38:22.546976 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:22.547014 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
I20260811 14:38:22.683856 27652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:22.683613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.67:46721)
0811 14:38:22.683764 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:22.683802 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260811 14:38:23.527551 27653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:23.527308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.65:60077)
0811 14:38:23.527456 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:23.527501 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":69,"threads_started":1}
I20260811 14:38:23.547824 27654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:23.547559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.66:58897)
0811 14:38:23.547696 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:23.547757 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":63,"threads_started":1}
I20260811 14:38:23.684522 27655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:23.684295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.67:52681)
0811 14:38:23.684430 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:23.684468 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260811 14:38:24.528366 27659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:24.528112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.65:53261)
0811 14:38:24.528266 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:24.528316 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260811 14:38:24.548594 27660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:24.548352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.66:55655)
0811 14:38:24.548500 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:24.548544 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260811 14:38:24.685349 27661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:24.685097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.67:39575)
0811 14:38:24.685252 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:24.685297 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
I20260811 14:38:25.469409 27663 raft_consensus.cc:482] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:38:25.469583 27663 raft_consensus.cc:504] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba71ee3655164ecba3f4063134037551" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 46803 } } peers { permanent_uuid: "6aad9e62431c49b9b115d08925eae94e" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37569 } } peers { permanent_uuid: "56f4d08c06f146a89480fa9e5a591c2d" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35451 } }
I20260811 14:38:25.469919 27663 leader_election.cc:288] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba71ee3655164ecba3f4063134037551 (127.2.103.66:46803), 6aad9e62431c49b9b115d08925eae94e (127.2.103.67:37569)
W20260811 14:38:25.470567 26978 leader_election.cc:341] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ba71ee3655164ecba3f4063134037551 (127.2.103.66:46803): Not found: Tablet not found: bee663da416245a0a2b2a719aa209288
W20260811 14:38:25.470752 26978 leader_election.cc:341] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6aad9e62431c49b9b115d08925eae94e (127.2.103.67:37569): Not found: Tablet not found: bee663da416245a0a2b2a719aa209288
I20260811 14:38:25.470809 26978 leader_election.cc:302] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56f4d08c06f146a89480fa9e5a591c2d; no voters: 6aad9e62431c49b9b115d08925eae94e, ba71ee3655164ecba3f4063134037551
I20260811 14:38:25.470966 27663 raft_consensus.cc:2726] T bee663da416245a0a2b2a719aa209288 P 56f4d08c06f146a89480fa9e5a591c2d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 14:38:25.529374 27669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:25.529097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.65:38005)
0811 14:38:25.529281 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:25.529321 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":84,"threads_started":1}
I20260811 14:38:25.549319 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:25.549121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.66:52409)
0811 14:38:25.549240 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:25.549271 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260811 14:38:25.685791 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 26899
I20260811 14:38:25.688796 27678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:25.686042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46077 (local address 127.2.103.67:34709)
0811 14:38:25.688682 (+ 2640us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:25.688735 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:46077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2614,"thread_start_us":58,"threads_started":1}
I20260811 14:38:25.703528 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 27176
I20260811 14:38:25.722327 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 27388
2026-08-11T14:38:25Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18704 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:38:25 dist-test-slave-twwt krb5kdc[27682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:38:25 dist-test-slave-twwt krb5kdc[27682](info): set up 2 sockets
Aug 11 14:38:25 dist-test-slave-twwt krb5kdc[27682](info): commencing operation
krb5kdc: starting...
W20260811 14:38:27.005826 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:38:27 dist-test-slave-twwt krb5kdc[27682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:38:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:38:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:38:27.138598 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:45907
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:45907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:38:27.224517 27703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:27.224786 27703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:27.224829 27703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:27.226042 27703 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 14:38:27.226086 27703 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 14:38:27.226100 27703 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 14:38:27.226112 27703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:38:27.226125 27703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:27.226138 27703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:27.226150 27703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:38:27.226207 27703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:38:27.227768 27703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:45907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:45907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:27.228063 27703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:27.228312 27703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:27.233588 27719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:27.233588 27722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:27.233695 27718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:27.233742 27703 server_base.cc:1034] running on GCE node
I20260811 14:38:27.234304 27703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:27.234712 27703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:27.235921 27703 hybrid_clock.cc:648] HybridClock initialized: now 1786459107235918 us; error 63 us; skew 500 ppm
Aug 11 14:38:27 dist-test-slave-twwt krb5kdc[27682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459107, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:27.238679 27703 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:38:27.239189 27703 webserver.cc:460] Webserver started at http://127.2.103.126:41727/ using document root <none> and password file <none>
I20260811 14:38:27.239454 27703 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:27.239501 27703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:27.239642 27703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:38:27.240761 27703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "96284e71ed334f7bb8a68dbf1109c88a"
format_stamp: "Formatted at 2026-08-11 14:38:27 on dist-test-slave-twwt"
server_key: "20e14ef4eeac1e096e227c2edd8450ac"
server_key_iv: "d1a7518123ff67dd508801ed5a238fa2"
server_key_version: "encryptionkey@0"
I20260811 14:38:27.241153 27703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "96284e71ed334f7bb8a68dbf1109c88a"
format_stamp: "Formatted at 2026-08-11 14:38:27 on dist-test-slave-twwt"
server_key: "20e14ef4eeac1e096e227c2edd8450ac"
server_key_iv: "d1a7518123ff67dd508801ed5a238fa2"
server_key_version: "encryptionkey@0"
I20260811 14:38:27.242655 27703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:38:27.243489 27732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:27.243686 27703 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:27.243800 27703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "96284e71ed334f7bb8a68dbf1109c88a"
format_stamp: "Formatted at 2026-08-11 14:38:27 on dist-test-slave-twwt"
server_key: "20e14ef4eeac1e096e227c2edd8450ac"
server_key_iv: "d1a7518123ff67dd508801ed5a238fa2"
server_key_version: "encryptionkey@0"
I20260811 14:38:27.243882 27703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:27.250571 27703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:27.252722 27703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:27.252892 27703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:27.257560 27703 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:45907
I20260811 14:38:27.257599 27821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:45907 every 8 connection(s)
I20260811 14:38:27.258009 27703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:38:27.259099 27822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:27.261817 27822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a: Bootstrap starting.
I20260811 14:38:27.262519 27822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:27.262912 27822 log.cc:824] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:27.263679 27822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a: No bootstrap required, opened a new log
I20260811 14:38:27.264568 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 27703
I20260811 14:38:27.264650 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:38:27.264849 2461 external_mini_cluster.cc:1428] Setting key 0acb64dec486342344085604f7ae7a86
I20260811 14:38:27.265941 27822 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96284e71ed334f7bb8a68dbf1109c88a" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45907 } }
I20260811 14:38:27.266139 27822 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:27.266261 27822 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96284e71ed334f7bb8a68dbf1109c88a, State: Initialized, Role: FOLLOWER
I20260811 14:38:27.266433 27822 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96284e71ed334f7bb8a68dbf1109c88a" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45907 } }
I20260811 14:38:27.266539 27822 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:38:27.266578 27822 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:38:27.266724 27822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:38:27.267885 27822 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96284e71ed334f7bb8a68dbf1109c88a" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45907 } }
I20260811 14:38:27.267989 27822 leader_election.cc:302] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96284e71ed334f7bb8a68dbf1109c88a; no voters:
I20260811 14:38:27.268260 27822 leader_election.cc:288] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:38:27.268291 27830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:38:27.268416 27830 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 1 LEADER]: Becoming Leader. State: Replica: 96284e71ed334f7bb8a68dbf1109c88a, State: Running, Role: LEADER
I20260811 14:38:27.268575 27822 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:38:27.268572 27830 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96284e71ed334f7bb8a68dbf1109c88a" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45907 } }
I20260811 14:38:27.269259 27833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96284e71ed334f7bb8a68dbf1109c88a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96284e71ed334f7bb8a68dbf1109c88a" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45907 } } }
I20260811 14:38:27.269582 27833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [sys.catalog]: This master's current role is: LEADER
I20260811 14:38:27.269490 27831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96284e71ed334f7bb8a68dbf1109c88a. Latest consensus state: current_term: 1 leader_uuid: "96284e71ed334f7bb8a68dbf1109c88a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96284e71ed334f7bb8a68dbf1109c88a" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45907 } } }
I20260811 14:38:27.270033 27831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [sys.catalog]: This master's current role is: LEADER
W20260811 14:38:27.270926 27857 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:38:27.271008 27857 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:38:27.271217 27853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:38:27.271466 27853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:38:27.273279 27853 catalog_manager.cc:1347] Generated new cluster ID: b488d39b5e4e4f32b50b29b45721f534
I20260811 14:38:27.273355 27853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:38:27 dist-test-slave-twwt krb5kdc[27682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:38:27.278224 27829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:27.265448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38588 (local address 127.2.103.126:45907)
0811 14:38:27.265624 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:27.265632 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:27.265658 (+ 26us) server_negotiation.cc:407] Connection header received
0811 14:38:27.265911 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:27.265917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:27.265966 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:27.266080 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:27.267083 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.268052 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.269036 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.269658 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.269768 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:27.274051 (+ 4283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:27.274057 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:27.274057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:27.274070 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:27.276351 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:27.276849 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:27.276853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:27.276853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:27.276985 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:27.277228 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:27.277229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:27.277230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:27.277560 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:27.277715 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:27.277851 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 14:38:27.278005 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260811 14:38:27.280849 27853 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:38:27.281453 27853 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:38:27.285916 27853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a: Generated new TSK 0
I20260811 14:38:27.286126 27853 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
I20260811 14:38:27.328799 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:45907
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:38:27.420176 27867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:27.420392 27867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:27.420418 27867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:27.421598 27867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:27.421644 27867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:27.421671 27867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:38:27.423260 27867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:45907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:27.423525 27867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:27.423780 27867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:27.428989 27872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:27.429001 27877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:27.429001 27874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:27.429304 27867 server_base.cc:1034] running on GCE node
I20260811 14:38:27.429495 27867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:27.429859 27867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:27.431023 27867 hybrid_clock.cc:648] HybridClock initialized: now 1786459107431002 us; error 36 us; skew 500 ppm
Aug 11 14:38:27 dist-test-slave-twwt krb5kdc[27682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459107, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:27.433941 27867 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:38:27.434422 27867 webserver.cc:460] Webserver started at http://127.2.103.65:43887/ using document root <none> and password file <none>
I20260811 14:38:27.434742 27867 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:27.434819 27867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:27.434981 27867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:38:27.436091 27867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "d506e40242e7432bab279e86d17dc3eb"
format_stamp: "Formatted at 2026-08-11 14:38:27 on dist-test-slave-twwt"
server_key: "0cbb69ca4804f966464e1a39a470d804"
server_key_iv: "d6963dade0cf7805578561cb29c0d495"
server_key_version: "encryptionkey@0"
I20260811 14:38:27.436491 27867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "d506e40242e7432bab279e86d17dc3eb"
format_stamp: "Formatted at 2026-08-11 14:38:27 on dist-test-slave-twwt"
server_key: "0cbb69ca4804f966464e1a39a470d804"
server_key_iv: "d6963dade0cf7805578561cb29c0d495"
server_key_version: "encryptionkey@0"
I20260811 14:38:27.437878 27867 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:38:27.438714 27890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:27.438892 27867 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:27.438997 27867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "d506e40242e7432bab279e86d17dc3eb"
format_stamp: "Formatted at 2026-08-11 14:38:27 on dist-test-slave-twwt"
server_key: "0cbb69ca4804f966464e1a39a470d804"
server_key_iv: "d6963dade0cf7805578561cb29c0d495"
server_key_version: "encryptionkey@0"
I20260811 14:38:27.439085 27867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:27.449281 27867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:27.451599 27867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:27.451815 27867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:27.452086 27867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:38:27.452426 27867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:38:27.452483 27867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:27.452523 27867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:38:27.452553 27867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:27.458484 27867 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:34115
I20260811 14:38:27.458570 28067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:34115 every 8 connection(s)
I20260811 14:38:27.458952 27867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:38:27 dist-test-slave-twwt krb5kdc[27682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459107, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:38:27.465972 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 27867
I20260811 14:38:27.466101 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:38:27.466344 2461 external_mini_cluster.cc:1428] Setting key 269143e0622ed34c6c6430138e5af22e
I20260811 14:38:27.468832 27829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:27.459769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:54007 (local address 127.2.103.126:45907)
0811 14:38:27.459831 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:27.459834 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:27.460029 (+ 195us) server_negotiation.cc:407] Connection header received
0811 14:38:27.460524 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:27.460525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:27.460536 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:27.460592 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:27.461517 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.462018 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.462996 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.463199 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.463254 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:27.465477 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:27.465479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:27.465479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:27.465488 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:27.467355 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:27.467751 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:27.467752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:27.467753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:27.467817 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:27.468073 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:27.468074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:27.468075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:27.468234 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:27.468305 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:27.468581 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 14:38:27.468684 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 14:38:27.468993 28070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:27.459720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.65:54007)
0811 14:38:27.459900 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:27.459943 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:27.460440 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:27.460654 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:27.460657 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:27.461239 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:27.461440 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:27.461445 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.462105 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.462106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:27.462922 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:27.462924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.463299 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.463301 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:27.463415 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:27.463892 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:27.463901 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:27.465381 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:27.467443 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:27.467451 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:27.467453 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:27.467674 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:27.467888 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:27.467888 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:27.467889 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:27.467995 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:27.468317 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:27.468324 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:27.468444 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 14:38:27.468535 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 14:38:27.468726 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
I20260811 14:38:27.469696 28068 heartbeater.cc:347] Connected to a master server at 127.2.103.126:45907
I20260811 14:38:27.469839 28068 heartbeater.cc:464] Registering TS with master...
I20260811 14:38:27.470074 28068 heartbeater.cc:511] Master 127.2.103.126:45907 requested a full tablet report, sending...
I20260811 14:38:27.470734 27766 ts_manager.cc:194] Registered new tserver with Master: d506e40242e7432bab279e86d17dc3eb (127.2.103.65:34115)
I20260811 14:38:27.471349 27766 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:54007
WARNING: no policy specified for kudu/127.2.103.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
I20260811 14:38:27.495990 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:45907
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:38:27.583401 28073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:27.583626 28073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:27.583653 28073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:27.584895 28073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:27.584950 28073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:27.585011 28073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:38:27.586576 28073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:45907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:27.586903 28073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:27.587172 28073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:27.592558 28098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:27.592579 28073 server_base.cc:1034] running on GCE node
W20260811 14:38:27.592561 28089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:27.592558 28086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:27.593117 28073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:27.593398 28073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:27.594581 28073 hybrid_clock.cc:648] HybridClock initialized: now 1786459107594568 us; error 31 us; skew 500 ppm
Aug 11 14:38:27 dist-test-slave-twwt krb5kdc[27682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459107, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:27.597635 28073 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:38:27.598138 28073 webserver.cc:460] Webserver started at http://127.2.103.66:35039/ using document root <none> and password file <none>
I20260811 14:38:27.598388 28073 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:27.598465 28073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:27.598672 28073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:38:27.599830 28073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "695bbd4ab27d494b85d412e8a37bac91"
format_stamp: "Formatted at 2026-08-11 14:38:27 on dist-test-slave-twwt"
server_key: "53c4f43ebf59d9dc1a021169d9a4eeae"
server_key_iv: "02994e9dc309251ca98f9f2ce500f059"
server_key_version: "encryptionkey@0"
I20260811 14:38:27.600229 28073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "695bbd4ab27d494b85d412e8a37bac91"
format_stamp: "Formatted at 2026-08-11 14:38:27 on dist-test-slave-twwt"
server_key: "53c4f43ebf59d9dc1a021169d9a4eeae"
server_key_iv: "02994e9dc309251ca98f9f2ce500f059"
server_key_version: "encryptionkey@0"
I20260811 14:38:27.601684 28073 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:38:27.602574 28111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:27.602792 28073 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:38:27.602900 28073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "695bbd4ab27d494b85d412e8a37bac91"
format_stamp: "Formatted at 2026-08-11 14:38:27 on dist-test-slave-twwt"
server_key: "53c4f43ebf59d9dc1a021169d9a4eeae"
server_key_iv: "02994e9dc309251ca98f9f2ce500f059"
server_key_version: "encryptionkey@0"
I20260811 14:38:27.602999 28073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:27.621707 28073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:27.624394 28073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:27.624593 28073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:27.624863 28073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:38:27.625216 28073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:38:27.625273 28073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:27.625316 28073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:38:27.625360 28073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:27.631774 28073 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:40427
I20260811 14:38:27.631824 28294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:40427 every 8 connection(s)
I20260811 14:38:27.632195 28073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:38:27.632742 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 28073
I20260811 14:38:27.632856 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:38:27.633028 2461 external_mini_cluster.cc:1428] Setting key 79eede149573f3f630283b43f38ec484
Aug 11 14:38:27 dist-test-slave-twwt krb5kdc[27682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459107, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.103.67@KRBTEST.COM; defaulting to no policy
I20260811 14:38:27.643312 27829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:27.633643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:33373 (local address 127.2.103.126:45907)
0811 14:38:27.633689 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:27.633692 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:27.633983 (+ 291us) server_negotiation.cc:407] Connection header received
0811 14:38:27.634470 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:27.634471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:27.634483 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:27.634550 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:27.635620 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.636133 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.637169 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.637377 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.637470 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:27.640125 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:27.640127 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:27.640127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:27.640138 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:27.641630 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:27.642165 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:27.642166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:27.642167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:27.642233 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:27.642506 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:27.642508 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:27.642508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:27.642692 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:27.642771 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:27.643117 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 14:38:27.643216 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 14:38:27.643487 28297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:27.633641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.66:33373)
0811 14:38:27.633833 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:27.633882 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:27.634377 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:27.634657 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:27.634661 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:27.635291 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:27.635511 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:27.635516 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.636248 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.636249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:27.637072 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:27.637074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.637483 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.637484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:27.637618 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:27.638250 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:27.638262 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:27.640016 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:27.641741 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:27.641750 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:27.641753 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:27.642082 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:27.642311 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:27.642312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:27.642312 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:27.642435 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:27.642777 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:27.642784 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:27.642937 (+ 153us) client_negotiation.cc:769] Sending connection context
0811 14:38:27.643063 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 14:38:27.643239 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":93,"threads_started":1}
I20260811 14:38:27.643987 28295 heartbeater.cc:347] Connected to a master server at 127.2.103.126:45907
I20260811 14:38:27.644078 28295 heartbeater.cc:464] Registering TS with master...
I20260811 14:38:27.644263 28295 heartbeater.cc:511] Master 127.2.103.126:45907 requested a full tablet report, sending...
I20260811 14:38:27.644744 27766 ts_manager.cc:194] Registered new tserver with Master: 695bbd4ab27d494b85d412e8a37bac91 (127.2.103.66:40427)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.67@KRBTEST.COM" created.
I20260811 14:38:27.645304 27766 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:33373
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
I20260811 14:38:27.661593 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:45907
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:38:27.747735 28303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:27.747968 28303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:27.747993 28303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:27.749195 28303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:27.749241 28303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:27.749269 28303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:38:27.750831 28303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:45907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:27.751055 28303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:27.751250 28303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:27.756709 28320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:27.756744 28316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:27.756731 28315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:27.756980 28303 server_base.cc:1034] running on GCE node
I20260811 14:38:27.757238 28303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:27.757467 28303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:27.758671 28303 hybrid_clock.cc:648] HybridClock initialized: now 1786459107758639 us; error 39 us; skew 500 ppm
Aug 11 14:38:27 dist-test-slave-twwt krb5kdc[27682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459107, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:27.761497 28303 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:38:27.762035 28303 webserver.cc:460] Webserver started at http://127.2.103.67:38219/ using document root <none> and password file <none>
I20260811 14:38:27.762296 28303 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:27.762346 28303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:27.762493 28303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:38:27.763702 28303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8"
format_stamp: "Formatted at 2026-08-11 14:38:27 on dist-test-slave-twwt"
server_key: "e0ce1ca07fa6e32cde6a14c9f26109c8"
server_key_iv: "24405d0de88de0b19ef22f866eb78f8c"
server_key_version: "encryptionkey@0"
I20260811 14:38:27.764108 28303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8"
format_stamp: "Formatted at 2026-08-11 14:38:27 on dist-test-slave-twwt"
server_key: "e0ce1ca07fa6e32cde6a14c9f26109c8"
server_key_iv: "24405d0de88de0b19ef22f866eb78f8c"
server_key_version: "encryptionkey@0"
I20260811 14:38:27.765527 28303 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:38:27.766345 28329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:27.766548 28303 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:27.766690 28303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8"
format_stamp: "Formatted at 2026-08-11 14:38:27 on dist-test-slave-twwt"
server_key: "e0ce1ca07fa6e32cde6a14c9f26109c8"
server_key_iv: "24405d0de88de0b19ef22f866eb78f8c"
server_key_version: "encryptionkey@0"
I20260811 14:38:27.766776 28303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:27.792863 28303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:27.795274 28303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:27.795452 28303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:27.795702 28303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:38:27.796038 28303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:38:27.796098 28303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:27.796141 28303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:38:27.796167 28303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:27.802152 28303 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:43699
I20260811 14:38:27.802201 28505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:43699 every 8 connection(s)
I20260811 14:38:27.802534 28303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:38:27.807998 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 28303
I20260811 14:38:27.808128 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:38:27.808322 2461 external_mini_cluster.cc:1428] Setting key cae4368a558cc906f4403ee3d84b23e2
Aug 11 14:38:27 dist-test-slave-twwt krb5kdc[27682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459107, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:38:27.812538 27829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:27.803374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:51093 (local address 127.2.103.126:45907)
0811 14:38:27.803481 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:27.803484 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:27.803671 (+ 187us) server_negotiation.cc:407] Connection header received
0811 14:38:27.804246 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:27.804248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:27.804261 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:27.804314 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:27.805326 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.805893 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.806984 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.807193 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.807281 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:27.809861 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:27.809862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:27.809863 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:27.809870 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:27.811179 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:27.811620 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:27.811620 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:27.811621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:27.811678 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:27.811963 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:27.811964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:27.811964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:27.812076 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:27.812132 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:27.812403 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 14:38:27.812472 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 14:38:27.812790 28517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:27.803291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.67:51093)
0811 14:38:27.803540 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:27.803580 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:27.804122 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:27.804393 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:27.804397 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:27.805004 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:27.805206 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:27.805211 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.805994 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.805995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:27.806882 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:27.806885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.807294 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.807295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:27.807421 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:27.807960 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:27.807972 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:27.809779 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:27.811286 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:27.811294 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:27.811296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:27.811546 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:27.811743 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:27.811744 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:27.811744 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:27.811863 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:27.812140 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:27.812146 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:27.812280 (+ 134us) client_negotiation.cc:769] Sending connection context
0811 14:38:27.812372 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 14:38:27.812502 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":105,"threads_started":1}
I20260811 14:38:27.813318 28511 heartbeater.cc:347] Connected to a master server at 127.2.103.126:45907
I20260811 14:38:27.813426 28511 heartbeater.cc:464] Registering TS with master...
I20260811 14:38:27.813618 28511 heartbeater.cc:511] Master 127.2.103.126:45907 requested a full tablet report, sending...
I20260811 14:38:27.814046 27766 ts_manager.cc:194] Registered new tserver with Master: ac27d7e4e1f3437ba9804c6cd3a5d9d8 (127.2.103.67:43699)
I20260811 14:38:27.814421 27766 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:51093
I20260811 14:38:27.820403 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:38:27.828410 27829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:27.821880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38604 (local address 127.2.103.126:45907)
0811 14:38:27.821923 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:27.821930 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:27.822093 (+ 163us) server_negotiation.cc:407] Connection header received
0811 14:38:27.822176 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:27.822177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:27.822191 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:27.822256 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:27.822949 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.823441 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.824192 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.824383 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.824458 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:27.825678 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:27.825680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:27.825680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:27.825690 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:27.827110 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:27.827523 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:27.827524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:27.827524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:27.827587 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:27.827807 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:27.827808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:27.827808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:27.827965 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:27.828075 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:27.828209 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 14:38:27.828298 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 14:38:27.830336 27766 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:38:27.832002 27766 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:38:27.839437 28539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:27.835484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36030 (local address 127.2.103.67:43699)
0811 14:38:27.835776 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:27.835781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:27.835791 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:38:27.835828 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:27.835830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:27.835846 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:27.835925 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:27.836044 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.837046 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.838182 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.838981 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.839066 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:27.839139 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:27.839300 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 14:38:27.839344 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":99,"threads_started":1}
I20260811 14:38:27.839438 28537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:27.834868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:40427 (local address 127.0.0.1:47132)
0811 14:38:27.835175 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:27.835186 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:27.835242 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:27.835489 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:27.835493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:27.835495 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:27.835573 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:27.835575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.836928 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.836928 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:27.838246 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:27.838249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.839149 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.839150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:27.839229 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:27.839234 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:38:27.839298 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 14:38:27.839347 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":74,"thread_start_us":116,"threads_started":1}
I20260811 14:38:27.839457 28532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:27.835032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:43699 (local address 127.0.0.1:36030)
0811 14:38:27.835312 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:27.835323 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:27.835364 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:27.835936 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:27.835937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:27.835939 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:27.835986 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:27.835988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.837151 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.837153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:27.838079 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:27.838082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.839080 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.839081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:27.839165 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:27.839182 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 14:38:27.839271 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 14:38:27.839334 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":99,"threads_started":1}
I20260811 14:38:27.839954 28536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:27.834996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47132 (local address 127.2.103.66:40427)
0811 14:38:27.835210 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:27.835215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:27.835231 (+ 16us) server_negotiation.cc:407] Connection header received
0811 14:38:27.835340 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:27.835343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:27.835362 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:27.835477 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:27.835664 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.836827 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.838339 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.839050 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.839680 (+ 630us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:27.839755 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:27.839785 (+ 30us) server_negotiation.cc:299] Negotiation successful
0811 14:38:27.839855 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":95,"threads_started":1}
I20260811 14:38:27.840492 28407 tablet_service.cc:1467] Processing CreateTablet for tablet 2b8ea27353fc4a73bc5a3282703c9a4e (DEFAULT_TABLE table=test-table [id=5f7a97667ad34d6493df8534941da099]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:38:27.840862 28407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b8ea27353fc4a73bc5a3282703c9a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:27.840790 28184 tablet_service.cc:1467] Processing CreateTablet for tablet 2b8ea27353fc4a73bc5a3282703c9a4e (DEFAULT_TABLE table=test-table [id=5f7a97667ad34d6493df8534941da099]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:38:27.841137 28184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b8ea27353fc4a73bc5a3282703c9a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:27.844075 28549 tablet_bootstrap.cc:492] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8: Bootstrap starting.
I20260811 14:38:27.844208 28534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:27.834855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:34115 (local address 127.0.0.1:53154)
0811 14:38:27.835067 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:27.835114 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:27.835193 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:27.835815 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:27.835816 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:27.835818 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:27.835908 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:27.835911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.836844 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.836845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:27.838261 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:27.838263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.839888 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.839890 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:27.839961 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:27.839964 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:38:27.840015 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 14:38:27.844115 (+ 4100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":33,"thread_start_us":60,"threads_started":1}
I20260811 14:38:27.844583 28538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:27.835246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53154 (local address 127.2.103.65:34115)
0811 14:38:27.835410 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:27.835416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:27.835426 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:38:27.835473 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:27.835619 (+ 146us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:27.835644 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:27.835726 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:27.836000 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.836734 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.838724 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.839393 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.839462 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:27.839519 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:27.844445 (+ 4926us) server_negotiation.cc:299] Negotiation successful
0811 14:38:27.844510 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":100,"threads_started":1}
I20260811 14:38:27.844821 28549 tablet_bootstrap.cc:654] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:27.845428 28549 log.cc:824] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:27.845412 27971 tablet_service.cc:1467] Processing CreateTablet for tablet 2b8ea27353fc4a73bc5a3282703c9a4e (DEFAULT_TABLE table=test-table [id=5f7a97667ad34d6493df8534941da099]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:38:27.845794 27971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b8ea27353fc4a73bc5a3282703c9a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:27.845830 28541 tablet_bootstrap.cc:492] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91: Bootstrap starting.
I20260811 14:38:27.846309 28549 tablet_bootstrap.cc:492] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8: No bootstrap required, opened a new log
I20260811 14:38:27.846387 28549 ts_tablet_manager.cc:1397] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:38:27.846824 28541 tablet_bootstrap.cc:654] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:27.847440 28541 log.cc:824] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:27.848359 28541 tablet_bootstrap.cc:492] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91: No bootstrap required, opened a new log
I20260811 14:38:27.848443 28541 ts_tablet_manager.cc:1397] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 14:38:27.848369 28549 raft_consensus.cc:348] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } } peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } }
I20260811 14:38:27.848577 28549 raft_consensus.cc:374] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:27.848623 28549 raft_consensus.cc:729] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac27d7e4e1f3437ba9804c6cd3a5d9d8, State: Initialized, Role: FOLLOWER
I20260811 14:38:27.848742 28549 consensus_queue.cc:260] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } } peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } }
I20260811 14:38:27.849053 28549 ts_tablet_manager.cc:1428] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 14:38:27.849195 28511 heartbeater.cc:503] Master 127.2.103.126:45907 was elected leader, sending a full tablet report...
I20260811 14:38:27.849553 28556 tablet_bootstrap.cc:492] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb: Bootstrap starting.
I20260811 14:38:27.850211 28556 tablet_bootstrap.cc:654] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:27.850922 28556 log.cc:824] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:27.850811 28541 raft_consensus.cc:348] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } } peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } }
I20260811 14:38:27.851022 28541 raft_consensus.cc:374] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:27.851076 28541 raft_consensus.cc:729] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 695bbd4ab27d494b85d412e8a37bac91, State: Initialized, Role: FOLLOWER
I20260811 14:38:27.851202 28541 consensus_queue.cc:260] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } } peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } }
I20260811 14:38:27.851532 28541 ts_tablet_manager.cc:1428] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:38:27.851569 28295 heartbeater.cc:503] Master 127.2.103.126:45907 was elected leader, sending a full tablet report...
I20260811 14:38:27.851830 28556 tablet_bootstrap.cc:492] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb: No bootstrap required, opened a new log
I20260811 14:38:27.851938 28556 ts_tablet_manager.cc:1397] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 14:38:27.853506 28556 raft_consensus.cc:348] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } } peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } }
I20260811 14:38:27.853709 28556 raft_consensus.cc:374] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:27.853771 28556 raft_consensus.cc:729] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d506e40242e7432bab279e86d17dc3eb, State: Initialized, Role: FOLLOWER
I20260811 14:38:27.853900 28556 consensus_queue.cc:260] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } } peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } }
I20260811 14:38:27.854192 28556 ts_tablet_manager.cc:1428] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 14:38:27.854225 28068 heartbeater.cc:503] Master 127.2.103.126:45907 was elected leader, sending a full tablet report...
I20260811 14:38:27.874308 28552 raft_consensus.cc:482] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:38:27.874459 28552 raft_consensus.cc:504] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } } peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } }
I20260811 14:38:27.874882 28552 leader_election.cc:288] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d506e40242e7432bab279e86d17dc3eb (127.2.103.65:34115), 695bbd4ab27d494b85d412e8a37bac91 (127.2.103.66:40427)
I20260811 14:38:27.879230 28536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:27.875168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:56595 (local address 127.2.103.66:40427)
0811 14:38:27.875233 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:27.875235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:27.875247 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:38:27.875280 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:27.875281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:27.875292 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:27.875371 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:27.875676 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.876602 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.878191 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.878808 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.878892 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:27.878902 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:27.879114 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 14:38:27.879157 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 14:38:27.879230 28538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:27.875128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:41229 (local address 127.2.103.65:34115)
0811 14:38:27.875185 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:27.875187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:27.875661 (+ 474us) server_negotiation.cc:407] Connection header received
0811 14:38:27.875702 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:27.875703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:27.875713 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:27.875777 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:27.875923 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.876696 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.878191 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.878808 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.878892 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:27.878902 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:27.879115 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 14:38:27.879157 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 14:38:27.879249 28517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:27.875067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:40427 (local address 127.2.103.67:56595)
0811 14:38:27.875141 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:27.875154 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:27.875201 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:27.875423 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:27.875425 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:27.875428 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:27.875508 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:27.875510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.876700 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.876701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:27.878069 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:27.878072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.878924 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.878926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:27.879022 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:27.879024 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:38:27.879094 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 14:38:27.879142 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":101}
I20260811 14:38:27.879271 28565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:27.875067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:34115 (local address 127.2.103.67:41229)
0811 14:38:27.875450 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:27.875461 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:27.875645 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:27.875784 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:27.875785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:27.875786 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:27.875845 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:27.875846 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.876780 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.876781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:27.878069 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:27.878072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.878924 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.878926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:27.879017 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:27.879022 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:38:27.879094 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 14:38:27.879142 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":109,"thread_start_us":185,"threads_started":1}
I20260811 14:38:27.879616 28221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b8ea27353fc4a73bc5a3282703c9a4e" candidate_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "695bbd4ab27d494b85d412e8a37bac91" is_pre_election: true
I20260811 14:38:27.879758 28221 raft_consensus.cc:2454] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac27d7e4e1f3437ba9804c6cd3a5d9d8 in term 0.
I20260811 14:38:27.880028 28000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b8ea27353fc4a73bc5a3282703c9a4e" candidate_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d506e40242e7432bab279e86d17dc3eb" is_pre_election: true
I20260811 14:38:27.880049 28332 leader_election.cc:302] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 695bbd4ab27d494b85d412e8a37bac91, ac27d7e4e1f3437ba9804c6cd3a5d9d8; no voters:
I20260811 14:38:27.880194 28000 raft_consensus.cc:2454] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac27d7e4e1f3437ba9804c6cd3a5d9d8 in term 0.
I20260811 14:38:27.880223 28552 raft_consensus.cc:2781] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:38:27.880301 28552 raft_consensus.cc:482] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:38:27.880329 28552 raft_consensus.cc:3037] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:38:27.881178 28552 raft_consensus.cc:504] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } } peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } }
I20260811 14:38:27.881349 28552 leader_election.cc:288] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [CANDIDATE]: Term 1 election: Requested vote from peers d506e40242e7432bab279e86d17dc3eb (127.2.103.65:34115), 695bbd4ab27d494b85d412e8a37bac91 (127.2.103.66:40427)
I20260811 14:38:27.881538 28221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b8ea27353fc4a73bc5a3282703c9a4e" candidate_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "695bbd4ab27d494b85d412e8a37bac91"
I20260811 14:38:27.881661 28221 raft_consensus.cc:3037] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:38:27.881726 28000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b8ea27353fc4a73bc5a3282703c9a4e" candidate_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d506e40242e7432bab279e86d17dc3eb"
I20260811 14:38:27.881836 28000 raft_consensus.cc:3037] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:38:27.882496 28221 raft_consensus.cc:2454] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac27d7e4e1f3437ba9804c6cd3a5d9d8 in term 1.
I20260811 14:38:27.882707 28332 leader_election.cc:302] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 695bbd4ab27d494b85d412e8a37bac91, ac27d7e4e1f3437ba9804c6cd3a5d9d8; no voters:
W20260811 14:38:27.882731 28296 tablet.cc:2369] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:38:27.882853 28000 raft_consensus.cc:2454] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac27d7e4e1f3437ba9804c6cd3a5d9d8 in term 1.
I20260811 14:38:27.882855 28552 raft_consensus.cc:2781] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:38:27.883059 28552 raft_consensus.cc:686] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 1 LEADER]: Becoming Leader. State: Replica: ac27d7e4e1f3437ba9804c6cd3a5d9d8, State: Running, Role: LEADER
I20260811 14:38:27.883180 28552 consensus_queue.cc:237] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } } peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } }
I20260811 14:38:27.884160 27766 catalog_manager.cc:5495] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac27d7e4e1f3437ba9804c6cd3a5d9d8 (127.2.103.67). New cstate: current_term: 1 leader_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } health_report { overall_health: HEALTHY } } }
I20260811 14:38:27.897255 28539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:27.894987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36046 (local address 127.2.103.67:43699)
0811 14:38:27.895052 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:27.895054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:27.895061 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:38:27.895097 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:27.895098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:27.895116 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:27.895183 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:38:27.895370 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.895839 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.896598 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:27.896776 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:27.896826 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:27.896967 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:38:27.897067 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:38:27.897129 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:27.897176 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 14:38:27.897194 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 14:38:27.899268 28000 raft_consensus.cc:1261] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Refusing update from remote peer ac27d7e4e1f3437ba9804c6cd3a5d9d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:38:27.899279 28221 raft_consensus.cc:1261] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91 [term 1 FOLLOWER]: Refusing update from remote peer ac27d7e4e1f3437ba9804c6cd3a5d9d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:38:27.899648 28552 consensus_queue.cc:1035] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:38:27.899825 28566 consensus_queue.cc:1035] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:38:27.903544 28572 mvcc.cc:205] Tried to move back new op lower bound from 7317336505952210944 to 7317336505890250752. Current Snapshot: MvccSnapshot[applied={T|T < 7317336505952210944 or (T in {7317336505952210944})}]
I20260811 14:38:27.904505 27766 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:38:27.904608 27766 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:38:27.905416 27766 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a: Sending DeleteTablet for 3 replicas of tablet 2b8ea27353fc4a73bc5a3282703c9a4e
I20260811 14:38:27.905773 27971 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b8ea27353fc4a73bc5a3282703c9a4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:38:27 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:53154
I20260811 14:38:27.905776 28184 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b8ea27353fc4a73bc5a3282703c9a4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:38:27 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:47132
I20260811 14:38:27.906035 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 27867
I20260811 14:38:27.906131 28407 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b8ea27353fc4a73bc5a3282703c9a4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:38:27 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:36030
I20260811 14:38:27.915316 28594 tablet_replica.cc:330] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8: stopping tablet replica
I20260811 14:38:27.915488 28594 raft_consensus.cc:2229] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:38:27.915671 28594 raft_consensus.cc:2258] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:38:27.917541 28594 ts_tablet_manager.cc:1905] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:38:27.918685 28589 tablet_replica.cc:330] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91: stopping tablet replica
I20260811 14:38:27.918828 28589 raft_consensus.cc:2229] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:38:27.918972 28589 raft_consensus.cc:2258] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:38:27.919440 28589 ts_tablet_manager.cc:1905] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:38:27.919857 28594 ts_tablet_manager.cc:1918] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:38:27.919930 28594 log.cc:1196] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/2b8ea27353fc4a73bc5a3282703c9a4e
I20260811 14:38:27.920221 28594 ts_tablet_manager.cc:1939] T 2b8ea27353fc4a73bc5a3282703c9a4e P ac27d7e4e1f3437ba9804c6cd3a5d9d8: Deleting consensus metadata
I20260811 14:38:27.921175 28589 ts_tablet_manager.cc:1918] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:38:27.921257 28589 log.cc:1196] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/2b8ea27353fc4a73bc5a3282703c9a4e
I20260811 14:38:27.921555 28589 ts_tablet_manager.cc:1939] T 2b8ea27353fc4a73bc5a3282703c9a4e P 695bbd4ab27d494b85d412e8a37bac91: Deleting consensus metadata
I20260811 14:38:27.922475 27739 catalog_manager.cc:4841] TS 695bbd4ab27d494b85d412e8a37bac91 (127.2.103.66:40427): tablet 2b8ea27353fc4a73bc5a3282703c9a4e (table test-table [id=5f7a97667ad34d6493df8534941da099]) successfully deleted
I20260811 14:38:27.923369 27733 catalog_manager.cc:4841] TS ac27d7e4e1f3437ba9804c6cd3a5d9d8 (127.2.103.67:43699): tablet 2b8ea27353fc4a73bc5a3282703c9a4e (table test-table [id=5f7a97667ad34d6493df8534941da099]) successfully deleted
W20260811 14:38:27.928568 27741 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.103.65:34115 (error 108)
I20260811 14:38:27.928886 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 28073
I20260811 14:38:27.939011 28534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:27.938814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:34115 (local address 127.0.0.1:53162)
0811 14:38:27.938892 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:27.938962 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.65:34115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 14:38:27.939182 27741 catalog_manager.cc:4584] TS d506e40242e7432bab279e86d17dc3eb (127.2.103.65:34115): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2b8ea27353fc4a73bc5a3282703c9a4e: Network error: Client connection negotiation failed: client connection to 127.2.103.65:34115: connect: Connection refused (error 111)
I20260811 14:38:27.948820 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 28303
I20260811 14:38:27.958771 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 27703
I20260811 14:38:47.969889 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:45907
--webserver_interface=127.2.103.126
--webserver_port=41727
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:45907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:38:48.063109 28596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:48.063335 28596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:48.063357 28596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:48.064445 28596 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 14:38:48.064477 28596 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 14:38:48.064491 28596 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 14:38:48.064502 28596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:38:48.064515 28596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:48.064528 28596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:48.064540 28596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:38:48.064553 28596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:38:48.066118 28596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:45907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:45907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=41727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:48.066386 28596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:48.066684 28596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:48.076874 28603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:48.076964 28596 server_base.cc:1034] running on GCE node
W20260811 14:38:48.076854 28605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:48.076879 28612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:48.077405 28596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:48.077782 28596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:48.078967 28596 hybrid_clock.cc:648] HybridClock initialized: now 1786459128078956 us; error 48 us; skew 500 ppm
Aug 11 14:38:48 dist-test-slave-twwt krb5kdc[27682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459128, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:48.087690 28596 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:38:48.088187 28596 webserver.cc:460] Webserver started at http://127.2.103.126:41727/ using document root <none> and password file <none>
I20260811 14:38:48.088454 28596 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:48.088543 28596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:48.090041 28596 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:38:48.090960 28621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:48.091132 28596 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:48.091255 28596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "96284e71ed334f7bb8a68dbf1109c88a"
format_stamp: "Formatted at 2026-08-11 14:38:27 on dist-test-slave-twwt"
server_key: "20e14ef4eeac1e096e227c2edd8450ac"
server_key_iv: "d1a7518123ff67dd508801ed5a238fa2"
server_key_version: "encryptionkey@0"
I20260811 14:38:48.091696 28596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:48.108693 28596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:48.111307 28596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:48.111491 28596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:48.116277 28596 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:45907
I20260811 14:38:48.116318 28706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:45907 every 8 connection(s)
I20260811 14:38:48.116797 28596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:38:48.117201 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 28596
I20260811 14:38:48.117506 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:34115
--local_ip_for_outbound_sockets=127.2.103.65
--tserver_master_addrs=127.2.103.126:45907
--webserver_port=43887
--webserver_interface=127.2.103.65
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:38:48.118592 28707 sys_catalog.cc:263] Verifying existing consensus state
I20260811 14:38:48.119365 28707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a: Bootstrap starting.
I20260811 14:38:48.128522 28707 log.cc:824] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:48.130659 28707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:38:48.131012 28707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a: Bootstrap complete.
I20260811 14:38:48.133487 28707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96284e71ed334f7bb8a68dbf1109c88a" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45907 } }
I20260811 14:38:48.133774 28707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96284e71ed334f7bb8a68dbf1109c88a, State: Initialized, Role: FOLLOWER
I20260811 14:38:48.133966 28707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96284e71ed334f7bb8a68dbf1109c88a" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45907 } }
I20260811 14:38:48.134057 28707 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:38:48.134131 28707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:38:48.134184 28707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 1 FOLLOWER]: Advancing to term 2
I20260811 14:38:48.135365 28707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96284e71ed334f7bb8a68dbf1109c88a" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45907 } }
I20260811 14:38:48.135492 28707 leader_election.cc:302] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96284e71ed334f7bb8a68dbf1109c88a; no voters:
I20260811 14:38:48.135696 28707 leader_election.cc:288] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 14:38:48.135736 28712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 2 FOLLOWER]: Leader election won for term 2
I20260811 14:38:48.135869 28712 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 2 LEADER]: Becoming Leader. State: Replica: 96284e71ed334f7bb8a68dbf1109c88a, State: Running, Role: LEADER
I20260811 14:38:48.135965 28712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96284e71ed334f7bb8a68dbf1109c88a" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45907 } }
I20260811 14:38:48.136003 28707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:38:48.136240 28713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "96284e71ed334f7bb8a68dbf1109c88a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96284e71ed334f7bb8a68dbf1109c88a" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45907 } } }
I20260811 14:38:48.136240 28712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96284e71ed334f7bb8a68dbf1109c88a. Latest consensus state: current_term: 2 leader_uuid: "96284e71ed334f7bb8a68dbf1109c88a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96284e71ed334f7bb8a68dbf1109c88a" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45907 } } }
I20260811 14:38:48.136461 28712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [sys.catalog]: This master's current role is: LEADER
I20260811 14:38:48.136441 28713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [sys.catalog]: This master's current role is: LEADER
I20260811 14:38:48.136842 28723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:38:48.137734 28723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:38:48.137899 28723 catalog_manager.cc:1259] Loaded cluster ID: b488d39b5e4e4f32b50b29b45721f534
I20260811 14:38:48.138167 28723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:38:48.139446 28723 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:38:48.139670 28723 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a: Loaded TSK: 0
I20260811 14:38:48.156149 28723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a: Generated new TSK 1
I20260811 14:38:48.156275 28723 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 14:38:48.209422 28709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:48.209674 28709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:48.209714 28709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:48.210932 28709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:48.210999 28709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:48.211045 28709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:38:48.212592 28709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:34115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=43887
--tserver_master_addrs=127.2.103.126:45907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:48.212898 28709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:48.213143 28709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:48.218331 28747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:48.218365 28749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:48.218365 28745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:48.218629 28709 server_base.cc:1034] running on GCE node
I20260811 14:38:48.218859 28709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:48.219106 28709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:48.220316 28709 hybrid_clock.cc:648] HybridClock initialized: now 1786459128220297 us; error 32 us; skew 500 ppm
Aug 11 14:38:48 dist-test-slave-twwt krb5kdc[27682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459128, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:48.223232 28709 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:38:48.223829 28709 webserver.cc:460] Webserver started at http://127.2.103.65:43887/ using document root <none> and password file <none>
I20260811 14:38:48.224098 28709 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:48.224184 28709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:48.225694 28709 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:48.226492 28759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:48.226758 28709 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:48.226861 28709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "d506e40242e7432bab279e86d17dc3eb"
format_stamp: "Formatted at 2026-08-11 14:38:27 on dist-test-slave-twwt"
server_key: "0cbb69ca4804f966464e1a39a470d804"
server_key_iv: "d6963dade0cf7805578561cb29c0d495"
server_key_version: "encryptionkey@0"
I20260811 14:38:48.227334 28709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:48.257670 28709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:48.260393 28709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:48.260558 28709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:48.260807 28709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:38:48.261314 28771 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 14:38:48.263140 28709 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 14:38:48.263206 28709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 14:38:48.263242 28709 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 14:38:48.264138 28709 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 14:38:48.264184 28709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 14:38:48.264292 28771 tablet_bootstrap.cc:492] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb: Bootstrap starting.
I20260811 14:38:48.271724 28709 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:34115
I20260811 14:38:48.272090 28949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:34115 every 8 connection(s)
I20260811 14:38:48.272204 28709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:38:48.274320 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 28709
I20260811 14:38:48.274793 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:40427
--local_ip_for_outbound_sockets=127.2.103.66
--tserver_master_addrs=127.2.103.126:45907
--webserver_port=35039
--webserver_interface=127.2.103.66
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:38:48.276835 28771 log.cc:824] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:48.280606 28771 tablet_bootstrap.cc:492] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:38:48.280995 28771 tablet_bootstrap.cc:492] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb: Bootstrap complete.
I20260811 14:38:48.282112 28771 ts_tablet_manager.cc:1397] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.011s
I20260811 14:38:48.283699 28771 raft_consensus.cc:348] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } } peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } }
I20260811 14:38:48.284050 28771 raft_consensus.cc:729] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d506e40242e7432bab279e86d17dc3eb, State: Initialized, Role: FOLLOWER
I20260811 14:38:48.284261 28771 consensus_queue.cc:260] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } } peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } }
I20260811 14:38:48.284680 28771 ts_tablet_manager.cc:1428] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb: Time spent starting tablet: real 0.002s user 0.006s sys 0.000s
Aug 11 14:38:48 dist-test-slave-twwt krb5kdc[27682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459128, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:38:48.290167 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.273325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:37279 (local address 127.2.103.126:45907)
0811 14:38:48.274753 (+ 1428us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:48.274764 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:48.274812 (+ 48us) server_negotiation.cc:407] Connection header received
0811 14:38:48.277709 (+ 2897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:48.277716 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:48.277775 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:48.277944 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:48.278997 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.280719 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.282527 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.283262 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.283411 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:48.286239 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:48.286251 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:48.286251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:48.286263 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:48.288122 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:48.288644 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:48.288648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:48.288649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:48.288766 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:48.289069 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:48.289070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:48.289070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:48.289344 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:48.289451 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:48.289779 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 14:38:48.289938 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1369,"thread_start_us":76,"threads_started":1}
I20260811 14:38:48.290166 28964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.273053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.65:37279)
0811 14:38:48.273255 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:48.273321 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:48.277336 (+ 4015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:48.278000 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:48.278005 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:48.278485 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:48.278826 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:48.278836 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.280874 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.280875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:48.282407 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:48.282411 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.283412 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.283414 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:48.283580 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:48.284258 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:48.284271 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:48.286110 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:48.288239 (+ 2129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:48.288247 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:48.288249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:48.288548 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:48.288861 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:48.288862 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:48.288862 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:48.288989 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:48.289424 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:48.289433 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:48.289592 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 14:38:48.289715 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 14:38:48.289935 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
I20260811 14:38:48.290997 28962 heartbeater.cc:347] Connected to a master server at 127.2.103.126:45907
I20260811 14:38:48.291114 28962 heartbeater.cc:464] Registering TS with master...
I20260811 14:38:48.291307 28962 heartbeater.cc:511] Master 127.2.103.126:45907 requested a full tablet report, sending...
I20260811 14:38:48.292130 28652 ts_manager.cc:194] Registered new tserver with Master: d506e40242e7432bab279e86d17dc3eb (127.2.103.65:34115)
I20260811 14:38:48.293431 28652 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:37279
I20260811 14:38:48.294512 28962 heartbeater.cc:503] Master 127.2.103.126:45907 was elected leader, sending a full tablet report...
Aug 11 14:38:48 dist-test-slave-twwt krb5kdc[27682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459128, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for kudu/127.2.103.65@KRBTEST.COM
I20260811 14:38:48.302664 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.292907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:34115 (local address 127.0.0.1:49996)
0811 14:38:48.293238 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:48.293285 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:48.293370 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:48.293644 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:48.293649 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:48.294002 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:48.294068 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:48.294070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.295556 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.295557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:48.296510 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:48.296512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.297003 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.297004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:48.297090 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:48.297588 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:48.297593 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:48.299271 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:48.301284 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:48.301293 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:48.301296 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:48.301548 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:48.301800 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:48.301801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:48.301801 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:48.301873 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:48.302197 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:48.302205 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:48.302268 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 14:38:48.302371 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 14:38:48.302527 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":101,"threads_started":1}
I20260811 14:38:48.302686 28972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.292988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49996 (local address 127.2.103.65:34115)
0811 14:38:48.293163 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:48.293167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:48.293373 (+ 206us) server_negotiation.cc:407] Connection header received
0811 14:38:48.293469 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:48.293472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:48.293490 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:48.293572 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:48.294158 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.295449 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.296597 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.296921 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.296986 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:48.299379 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:48.299388 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:48.299388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:48.299396 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:48.301194 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:48.301647 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:48.301650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:48.301651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:48.301718 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:48.301942 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:48.301943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:48.301943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:48.302111 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:48.302207 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:48.302428 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 14:38:48.302534 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":89,"threads_started":1}
I20260811 14:38:48.303294 28835 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b8ea27353fc4a73bc5a3282703c9a4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:38:27 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:49996
I20260811 14:38:48.303884 28974 tablet_replica.cc:330] stopping tablet replica
I20260811 14:38:48.304013 28974 raft_consensus.cc:2229] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:38:48.304095 28974 raft_consensus.cc:2258] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:38:48.304442 28974 ts_tablet_manager.cc:1905] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:38:48.307008 28974 ts_tablet_manager.cc:1918] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:38:48.307091 28974 log.cc:1196] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/wal/wals/2b8ea27353fc4a73bc5a3282703c9a4e
I20260811 14:38:48.307327 28974 ts_tablet_manager.cc:1939] T 2b8ea27353fc4a73bc5a3282703c9a4e P d506e40242e7432bab279e86d17dc3eb: Deleting consensus metadata
I20260811 14:38:48.307862 28626 catalog_manager.cc:4841] TS d506e40242e7432bab279e86d17dc3eb (127.2.103.65:34115): tablet 2b8ea27353fc4a73bc5a3282703c9a4e (table test-table [id=5f7a97667ad34d6493df8534941da099]) successfully deleted
W20260811 14:38:48.369688 28966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:48.369925 28966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:48.369949 28966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:48.371086 28966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:48.371122 28966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:48.371148 28966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:38:48.372586 28966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:40427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=35039
--tserver_master_addrs=127.2.103.126:45907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:48.372807 28966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:48.373018 28966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:48.378405 28982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:48.378592 28966 server_base.cc:1034] running on GCE node
W20260811 14:38:48.378419 28985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:48.378419 28983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:48.378978 28966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:48.379235 28966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:48.380460 28966 hybrid_clock.cc:648] HybridClock initialized: now 1786459128380448 us; error 32 us; skew 500 ppm
Aug 11 14:38:48 dist-test-slave-twwt krb5kdc[27682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459128, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:48.383220 28966 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:38:48.383673 28966 webserver.cc:460] Webserver started at http://127.2.103.66:35039/ using document root <none> and password file <none>
I20260811 14:38:48.383937 28966 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:48.384027 28966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:48.385434 28966 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:38:48.386129 29001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:48.386322 28966 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:48.386440 28966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "695bbd4ab27d494b85d412e8a37bac91"
format_stamp: "Formatted at 2026-08-11 14:38:27 on dist-test-slave-twwt"
server_key: "53c4f43ebf59d9dc1a021169d9a4eeae"
server_key_iv: "02994e9dc309251ca98f9f2ce500f059"
server_key_version: "encryptionkey@0"
I20260811 14:38:48.386830 28966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:48.399976 28966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:48.402364 28966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:48.402559 28966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:48.402848 28966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:38:48.403183 28966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:38:48.403239 28966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:48.403280 28966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:38:48.403304 28966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:48.409384 28966 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:40427
I20260811 14:38:48.409428 29208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:40427 every 8 connection(s)
I20260811 14:38:48.409798 28966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:38:48.412854 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 28966
I20260811 14:38:48.413185 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:43699
--local_ip_for_outbound_sockets=127.2.103.67
--tserver_master_addrs=127.2.103.126:45907
--webserver_port=38219
--webserver_interface=127.2.103.67
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 14:38:48 dist-test-slave-twwt krb5kdc[27682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459128, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:38:48.420204 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.410537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:60119 (local address 127.2.103.126:45907)
0811 14:38:48.410643 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:48.410645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:48.410848 (+ 203us) server_negotiation.cc:407] Connection header received
0811 14:38:48.411426 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:48.411427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:48.411447 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:48.411500 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:48.412516 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.413140 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.414240 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.414478 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.414555 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:48.417131 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:48.417132 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:48.417133 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:48.417143 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:48.418736 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:48.419173 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:48.419175 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:48.419175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:48.419242 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:48.419509 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:48.419510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:48.419510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:48.419635 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:48.419704 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:48.420006 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 14:38:48.420108 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 14:38:48.420411 29215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.410520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.66:60119)
0811 14:38:48.410726 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:48.410765 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:48.411305 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:48.411576 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:48.411581 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:48.412187 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:48.412404 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:48.412410 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.413236 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.413238 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:48.414139 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:48.414142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.414579 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.414580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:48.414739 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:48.415325 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:48.415337 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:48.417019 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:48.418838 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:48.418847 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:48.418850 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:48.419094 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:48.419311 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:48.419312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:48.419312 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:48.419435 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:48.419706 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:48.419714 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:48.419865 (+ 151us) client_negotiation.cc:769] Sending connection context
0811 14:38:48.419972 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 14:38:48.420145 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":124,"threads_started":1}
I20260811 14:38:48.420981 29213 heartbeater.cc:347] Connected to a master server at 127.2.103.126:45907
I20260811 14:38:48.421082 29213 heartbeater.cc:464] Registering TS with master...
I20260811 14:38:48.421290 29213 heartbeater.cc:511] Master 127.2.103.126:45907 requested a full tablet report, sending...
I20260811 14:38:48.421708 28652 ts_manager.cc:194] Registered new tserver with Master: 695bbd4ab27d494b85d412e8a37bac91 (127.2.103.66:40427)
I20260811 14:38:48.422184 28652 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:60119
W20260811 14:38:48.500072 29217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:38:48.500314 29217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:38:48.500352 29217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:38:48.501502 29217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:38:48.501565 29217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:38:48.501611 29217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:38:48.503188 29217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:43699
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=38219
--tserver_master_addrs=127.2.103.126:45907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:38:48.503461 29217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:38:48.503711 29217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:38:48.508991 29224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:48.509110 29217 server_base.cc:1034] running on GCE node
W20260811 14:38:48.508986 29228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:38:48.508996 29226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:38:48.509404 29217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:38:48.509703 29217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:38:48.510861 29217 hybrid_clock.cc:648] HybridClock initialized: now 1786459128510824 us; error 49 us; skew 500 ppm
Aug 11 14:38:48 dist-test-slave-twwt krb5kdc[27682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459128, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:38:48.513502 29217 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:38:48.513949 29217 webserver.cc:460] Webserver started at http://127.2.103.67:38219/ using document root <none> and password file <none>
I20260811 14:38:48.514204 29217 fs_manager.cc:359] Metadata directory not provided
I20260811 14:38:48.514286 29217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:38:48.515720 29217 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:48.516314 29237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:48.516499 29217 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:38:48.516594 29217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8"
format_stamp: "Formatted at 2026-08-11 14:38:27 on dist-test-slave-twwt"
server_key: "e0ce1ca07fa6e32cde6a14c9f26109c8"
server_key_iv: "24405d0de88de0b19ef22f866eb78f8c"
server_key_version: "encryptionkey@0"
I20260811 14:38:48.516942 29217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:38:48.543619 29217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:38:48.546171 29217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:38:48.546342 29217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:38:48.546630 29217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:38:48.546974 29217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:38:48.547029 29217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:48.547066 29217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:38:48.547109 29217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:38:48.552886 29217 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:43699
I20260811 14:38:48.552974 29435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:43699 every 8 connection(s)
I20260811 14:38:48.553344 29217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
Aug 11 14:38:48 dist-test-slave-twwt krb5kdc[27682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459128, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:38:48.560122 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 29217
I20260811 14:38:48.563161 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.554156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:56171 (local address 127.2.103.126:45907)
0811 14:38:48.554297 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:48.554299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:48.554446 (+ 147us) server_negotiation.cc:407] Connection header received
0811 14:38:48.555040 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:48.555041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:48.555053 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:48.555117 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:48.556039 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.556537 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.557517 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.557707 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.557800 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:48.560044 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:48.560046 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:48.560046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:48.560054 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:48.561705 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:48.562122 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:48.562123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:48.562124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:48.562192 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:48.562486 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:48.562487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:48.562487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:48.562590 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:48.562697 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:48.563025 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 14:38:48.563094 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 14:38:48.563418 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.554135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.67:56171)
0811 14:38:48.554309 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:48.554353 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:48.554914 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:48.555174 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:48.555178 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:48.555751 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:38:48.555957 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:48.555961 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.556615 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.556616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:48.557436 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:48.557438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.557784 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.557785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:48.557912 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:48.558422 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:38:48.558431 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:38:48.559924 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:38:48.561807 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:48.561813 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:48.561815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:48.562044 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:48.562259 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:38:48.562260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:38:48.562260 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:38:48.562416 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:38:48.562690 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:38:48.562697 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:38:48.562882 (+ 185us) client_negotiation.cc:769] Sending connection context
0811 14:38:48.563001 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 14:38:48.563171 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
I20260811 14:38:48.564009 29437 heartbeater.cc:347] Connected to a master server at 127.2.103.126:45907
I20260811 14:38:48.564100 29437 heartbeater.cc:464] Registering TS with master...
I20260811 14:38:48.564250 29437 heartbeater.cc:511] Master 127.2.103.126:45907 requested a full tablet report, sending...
I20260811 14:38:48.564636 28652 ts_manager.cc:194] Registered new tserver with Master: ac27d7e4e1f3437ba9804c6cd3a5d9d8 (127.2.103.67:43699)
I20260811 14:38:48.565013 28652 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:56171
I20260811 14:38:48.567864 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.560554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37380 (local address 127.2.103.126:45907)
0811 14:38:48.560732 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:48.560734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:48.560810 (+ 76us) server_negotiation.cc:407] Connection header received
0811 14:38:48.560869 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:48.560870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:48.560884 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:48.560954 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:48.561633 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.562268 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.563394 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.563721 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.563794 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:48.565239 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:48.565241 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:48.565241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:48.565248 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:48.566725 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:48.567079 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:48.567080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:48.567080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:48.567143 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:48.567352 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:48.567353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:48.567353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:48.567462 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:48.567641 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:48.567679 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 14:38:48.567761 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":100,"threads_started":1}
I20260811 14:38:48.568611 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:38:48.571256 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.568947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37396 (local address 127.2.103.126:45907)
0811 14:38:48.569006 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:48.569008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:48.569078 (+ 70us) server_negotiation.cc:407] Connection header received
0811 14:38:48.569134 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:48.569135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:48.569165 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:48.569228 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:38:48.569308 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.569736 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.570500 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.570784 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.570851 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:48.570990 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:38:48.571030 (+ 40us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 14:38:48.571183 (+ 153us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37396: token expired
Metrics: {"server-negotiator.queue_time_us":36}
W20260811 14:38:48.571318 29442 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37396: token expired
W20260811 14:38:48.571437 28522 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.2.103.126:45907)
I20260811 14:38:48.577435 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.571711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37412 (local address 127.2.103.126:45907)
0811 14:38:48.571751 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:48.571753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:48.571777 (+ 24us) server_negotiation.cc:407] Connection header received
0811 14:38:48.571795 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:48.571796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:48.571804 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:48.571897 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:38:48.572374 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.572820 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.573521 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.573701 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.573768 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:48.574967 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:38:48.574968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:38:48.574968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:38:48.574976 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:38:48.576320 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:48.576657 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:48.576658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:48.576659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:48.576718 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:38:48.576936 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:38:48.576937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:38:48.576937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:38:48.577047 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:38:48.577151 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:48.577255 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 14:38:48.577336 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:38:48.578697 28652 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:38:48.580349 28652 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:38:48.583194 28835 tablet_service.cc:1467] Processing CreateTablet for tablet 425395c1e4424c1e8d3ed1c0d495a62c (DEFAULT_TABLE table=test-table [id=666d8e2c02ac4ff1b0c2c4ce953b28d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:38:48.583455 28835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 425395c1e4424c1e8d3ed1c0d495a62c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:48.585083 28771 tablet_bootstrap.cc:492] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb: Bootstrap starting.
I20260811 14:38:48.585436 28771 tablet_bootstrap.cc:654] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:48.586371 28771 tablet_bootstrap.cc:492] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb: No bootstrap required, opened a new log
I20260811 14:38:48.586447 28771 ts_tablet_manager.cc:1397] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:38:48.586652 28771 raft_consensus.cc:348] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } } peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } }
I20260811 14:38:48.586747 28771 raft_consensus.cc:374] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:48.586804 28771 raft_consensus.cc:729] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d506e40242e7432bab279e86d17dc3eb, State: Initialized, Role: FOLLOWER
I20260811 14:38:48.586862 28771 consensus_queue.cc:260] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } } peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } }
I20260811 14:38:48.586962 28771 ts_tablet_manager.cc:1428] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:38:48.587325 29445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.582719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:40427 (local address 127.0.0.1:45402)
0811 14:38:48.583149 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:48.583162 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:48.583220 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:48.583504 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:48.583504 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:48.583506 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:48.583538 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:48.583540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.584647 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.584648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:48.585948 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:48.585950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.587014 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.587014 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:48.587095 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:48.587121 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 14:38:48.587178 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 14:38:48.587214 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":21,"thread_start_us":134,"threads_started":1}
I20260811 14:38:48.587342 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.582719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:43699 (local address 127.0.0.1:33796)
0811 14:38:48.582779 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:48.582790 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:48.582834 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:48.583462 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:48.583463 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:48.583465 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:48.583515 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:48.583517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.584817 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.584818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:48.585927 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:48.585931 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.587012 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.587014 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:48.587115 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:48.587127 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 14:38:48.587158 (+ 31us) client_negotiation.cc:240] Negotiation successful
0811 14:38:48.587221 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":7}
I20260811 14:38:48.587507 29448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.582931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33796 (local address 127.2.103.67:43699)
0811 14:38:48.583230 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:48.583235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:48.583248 (+ 13us) server_negotiation.cc:407] Connection header received
0811 14:38:48.583298 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:48.583300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:48.583318 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:48.583449 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:48.583585 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.584719 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.586101 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.586918 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.587268 (+ 350us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:48.587346 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:48.587379 (+ 33us) server_negotiation.cc:299] Negotiation successful
0811 14:38:48.587432 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":101,"threads_started":1}
I20260811 14:38:48.587568 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.582974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45402 (local address 127.2.103.66:40427)
0811 14:38:48.583190 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:48.583195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:48.583218 (+ 23us) server_negotiation.cc:407] Connection header received
0811 14:38:48.583303 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:48.583304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:48.583321 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:48.583399 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:48.583620 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.584554 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.586042 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.586835 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.586911 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:48.586981 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:48.587441 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 14:38:48.587484 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":115,"threads_started":1}
I20260811 14:38:48.588231 29092 tablet_service.cc:1467] Processing CreateTablet for tablet 425395c1e4424c1e8d3ed1c0d495a62c (DEFAULT_TABLE table=test-table [id=666d8e2c02ac4ff1b0c2c4ce953b28d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:38:48.588225 29320 tablet_service.cc:1467] Processing CreateTablet for tablet 425395c1e4424c1e8d3ed1c0d495a62c (DEFAULT_TABLE table=test-table [id=666d8e2c02ac4ff1b0c2c4ce953b28d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:38:48.588610 29320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 425395c1e4424c1e8d3ed1c0d495a62c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:48.588610 29092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 425395c1e4424c1e8d3ed1c0d495a62c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:38:48.592536 29455 tablet_bootstrap.cc:492] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91: Bootstrap starting.
I20260811 14:38:48.593257 29455 tablet_bootstrap.cc:654] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:48.593729 29455 log.cc:824] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:48.594167 29452 tablet_bootstrap.cc:492] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8: Bootstrap starting.
I20260811 14:38:48.594584 29455 tablet_bootstrap.cc:492] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91: No bootstrap required, opened a new log
I20260811 14:38:48.594697 29455 ts_tablet_manager.cc:1397] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:38:48.595048 29452 tablet_bootstrap.cc:654] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8: Neither blocks nor log segments found. Creating new log.
I20260811 14:38:48.595625 29452 log.cc:824] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8: Log is configured to *not* fsync() on all Append() calls
I20260811 14:38:48.596351 29452 tablet_bootstrap.cc:492] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8: No bootstrap required, opened a new log
I20260811 14:38:48.596408 29452 ts_tablet_manager.cc:1397] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:38:48.596489 29455 raft_consensus.cc:348] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } } peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } }
I20260811 14:38:48.596662 29455 raft_consensus.cc:374] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:48.596732 29455 raft_consensus.cc:729] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 695bbd4ab27d494b85d412e8a37bac91, State: Initialized, Role: FOLLOWER
I20260811 14:38:48.596896 29455 consensus_queue.cc:260] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } } peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } }
I20260811 14:38:48.597189 29455 ts_tablet_manager.cc:1428] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:38:48.597222 29213 heartbeater.cc:503] Master 127.2.103.126:45907 was elected leader, sending a full tablet report...
I20260811 14:38:48.598397 29452 raft_consensus.cc:348] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } } peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } }
I20260811 14:38:48.598579 29452 raft_consensus.cc:374] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:38:48.598662 29452 raft_consensus.cc:729] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac27d7e4e1f3437ba9804c6cd3a5d9d8, State: Initialized, Role: FOLLOWER
I20260811 14:38:48.598790 29452 consensus_queue.cc:260] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } } peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } }
I20260811 14:38:48.599092 29452 ts_tablet_manager.cc:1428] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:38:48.599112 29437 heartbeater.cc:503] Master 127.2.103.126:45907 was elected leader, sending a full tablet report...
W20260811 14:38:48.660392 29214 tablet.cc:2369] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:38:48.697381 29461 raft_consensus.cc:482] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:38:48.697561 29461 raft_consensus.cc:504] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } } peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } }
I20260811 14:38:48.697993 29461 leader_election.cc:288] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 695bbd4ab27d494b85d412e8a37bac91 (127.2.103.66:40427), d506e40242e7432bab279e86d17dc3eb (127.2.103.65:34115)
I20260811 14:38:48.702069 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.698248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:57473 (local address 127.2.103.66:40427)
0811 14:38:48.698294 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:48.698296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:48.698308 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:38:48.698398 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:48.698399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:48.698409 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:48.698483 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:48.698774 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.699527 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.701045 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.701607 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.701698 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:48.701709 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:48.701953 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 14:38:48.701999 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 14:38:48.702069 28972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.698622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:36127 (local address 127.2.103.65:34115)
0811 14:38:48.698690 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:48.698692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:48.698704 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:38:48.698726 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:48.698726 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:48.698786 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:48.698865 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:38:48.699053 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.699728 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.701050 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.701697 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.701770 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:48.701851 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:48.701952 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 14:38:48.701999 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 14:38:48.702065 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.698185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:40427 (local address 127.2.103.67:57473)
0811 14:38:48.698245 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:48.698271 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:48.698323 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:48.698512 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:48.698514 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:48.698521 (+ 7us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:48.698601 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:48.698650 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.699643 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.699644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:48.700903 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:48.700906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.701714 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.701716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:48.701838 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:48.701842 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:38:48.701913 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 14:38:48.701961 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":4}
I20260811 14:38:48.702071 29462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.698336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:34115 (local address 127.2.103.67:36127)
0811 14:38:48.698511 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:48.698522 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:38:48.698573 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:38:48.698889 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:38:48.698890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:38:48.698892 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:38:48.698968 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:48.698971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.699829 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.699830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:48.700958 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:38:48.700960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.701808 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.701809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:38:48.701871 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:48.701874 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:38:48.701939 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 14:38:48.701968 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":100,"threads_started":1}
I20260811 14:38:48.702466 28870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "425395c1e4424c1e8d3ed1c0d495a62c" candidate_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d506e40242e7432bab279e86d17dc3eb" is_pre_election: true
I20260811 14:38:48.702493 29126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "425395c1e4424c1e8d3ed1c0d495a62c" candidate_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "695bbd4ab27d494b85d412e8a37bac91" is_pre_election: true
I20260811 14:38:48.702674 28870 raft_consensus.cc:2454] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac27d7e4e1f3437ba9804c6cd3a5d9d8 in term 0.
I20260811 14:38:48.702688 29126 raft_consensus.cc:2454] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac27d7e4e1f3437ba9804c6cd3a5d9d8 in term 0.
I20260811 14:38:48.702941 29242 leader_election.cc:302] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 695bbd4ab27d494b85d412e8a37bac91, ac27d7e4e1f3437ba9804c6cd3a5d9d8; no voters:
I20260811 14:38:48.703121 29461 raft_consensus.cc:2781] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:38:48.703208 29461 raft_consensus.cc:482] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:38:48.703236 29461 raft_consensus.cc:3037] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:38:48.704063 29461 raft_consensus.cc:504] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } } peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } }
I20260811 14:38:48.704231 29461 leader_election.cc:288] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [CANDIDATE]: Term 1 election: Requested vote from peers 695bbd4ab27d494b85d412e8a37bac91 (127.2.103.66:40427), d506e40242e7432bab279e86d17dc3eb (127.2.103.65:34115)
I20260811 14:38:48.704401 29126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "425395c1e4424c1e8d3ed1c0d495a62c" candidate_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "695bbd4ab27d494b85d412e8a37bac91"
I20260811 14:38:48.704394 28870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "425395c1e4424c1e8d3ed1c0d495a62c" candidate_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d506e40242e7432bab279e86d17dc3eb"
I20260811 14:38:48.704484 29126 raft_consensus.cc:3037] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:38:48.704501 28870 raft_consensus.cc:3037] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:38:48.705219 29126 raft_consensus.cc:2454] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac27d7e4e1f3437ba9804c6cd3a5d9d8 in term 1.
I20260811 14:38:48.705410 28870 raft_consensus.cc:2454] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac27d7e4e1f3437ba9804c6cd3a5d9d8 in term 1.
I20260811 14:38:48.705400 29242 leader_election.cc:302] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 695bbd4ab27d494b85d412e8a37bac91, ac27d7e4e1f3437ba9804c6cd3a5d9d8; no voters:
I20260811 14:38:48.705571 29461 raft_consensus.cc:2781] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:38:48.705770 29461 raft_consensus.cc:686] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 1 LEADER]: Becoming Leader. State: Replica: ac27d7e4e1f3437ba9804c6cd3a5d9d8, State: Running, Role: LEADER
I20260811 14:38:48.705878 29461 consensus_queue.cc:237] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } } peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } }
I20260811 14:38:48.706790 28652 catalog_manager.cc:5495] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac27d7e4e1f3437ba9804c6cd3a5d9d8 (127.2.103.67). New cstate: current_term: 1 leader_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } health_report { overall_health: UNKNOWN } } }
I20260811 14:38:48.732661 29448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.730012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33810 (local address 127.2.103.67:43699)
0811 14:38:48.730073 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 14:38:48.730075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:38:48.730087 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:38:48.730178 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:38:48.730179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:38:48.730203 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:38:48.730283 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:38:48.730436 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.730996 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.731845 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:38:48.732042 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:38:48.732115 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:38:48.732285 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:38:48.732389 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:38:48.732473 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 14:38:48.732509 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 14:38:48.732531 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 14:38:48.734530 29126 raft_consensus.cc:1261] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91 [term 1 FOLLOWER]: Refusing update from remote peer ac27d7e4e1f3437ba9804c6cd3a5d9d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:38:48.734647 28870 raft_consensus.cc:1261] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Refusing update from remote peer ac27d7e4e1f3437ba9804c6cd3a5d9d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:38:48.735029 29464 consensus_queue.cc:1035] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:38:48.735157 29461 consensus_queue.cc:1035] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:38:48.737844 29469 mvcc.cc:205] Tried to move back new op lower bound from 7317336591293788160 to 7317336591179993088. Current Snapshot: MvccSnapshot[applied={T|T < 7317336591293788160}]
I20260811 14:38:48.740311 28652 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:38:48.740417 28652 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:38:48.741127 28652 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a: Sending DeleteTablet for 3 replicas of tablet 425395c1e4424c1e8d3ed1c0d495a62c
I20260811 14:38:48.741460 28835 tablet_service.cc:1514] Processing DeleteTablet for tablet 425395c1e4424c1e8d3ed1c0d495a62c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:38:48 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:49996
I20260811 14:38:48.741472 29320 tablet_service.cc:1514] Processing DeleteTablet for tablet 425395c1e4424c1e8d3ed1c0d495a62c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:38:48 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:33796
I20260811 14:38:48.741559 28974 tablet_replica.cc:330] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb: stopping tablet replica
I20260811 14:38:48.741609 28974 raft_consensus.cc:2229] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:38:48.741595 29092 tablet_service.cc:1514] Processing DeleteTablet for tablet 425395c1e4424c1e8d3ed1c0d495a62c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:38:48 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:45402
I20260811 14:38:48.741652 28974 raft_consensus.cc:2258] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:38:48.741822 29480 tablet_replica.cc:330] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8: stopping tablet replica
I20260811 14:38:48.741920 29480 raft_consensus.cc:2229] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:38:48.741935 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 28709
I20260811 14:38:48.742064 29480 raft_consensus.cc:2258] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:38:48.742090 28974 ts_tablet_manager.cc:1905] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:38:48.742540 29480 ts_tablet_manager.cc:1905] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:38:48.744855 29480 ts_tablet_manager.cc:1918] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:38:48.744952 29480 log.cc:1196] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/425395c1e4424c1e8d3ed1c0d495a62c
I20260811 14:38:48.745316 29480 ts_tablet_manager.cc:1939] T 425395c1e4424c1e8d3ed1c0d495a62c P ac27d7e4e1f3437ba9804c6cd3a5d9d8: Deleting consensus metadata
I20260811 14:38:48.745822 28622 catalog_manager.cc:4841] TS ac27d7e4e1f3437ba9804c6cd3a5d9d8 (127.2.103.67:43699): tablet 425395c1e4424c1e8d3ed1c0d495a62c (table test-table [id=666d8e2c02ac4ff1b0c2c4ce953b28d1]) successfully deleted
I20260811 14:38:48.746690 29481 tablet_replica.cc:330] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91: stopping tablet replica
I20260811 14:38:48.746851 29481 raft_consensus.cc:2229] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:38:48.747000 29481 raft_consensus.cc:2258] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:38:48.747550 29481 ts_tablet_manager.cc:1905] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:38:48.749222 29481 ts_tablet_manager.cc:1918] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:38:48.749329 29481 log.cc:1196] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/425395c1e4424c1e8d3ed1c0d495a62c
I20260811 14:38:48.749647 29481 ts_tablet_manager.cc:1939] T 425395c1e4424c1e8d3ed1c0d495a62c P 695bbd4ab27d494b85d412e8a37bac91: Deleting consensus metadata
I20260811 14:38:48.751667 28625 catalog_manager.cc:4841] TS 695bbd4ab27d494b85d412e8a37bac91 (127.2.103.66:40427): tablet 425395c1e4424c1e8d3ed1c0d495a62c (table test-table [id=666d8e2c02ac4ff1b0c2c4ce953b28d1]) successfully deleted
W20260811 14:38:48.758059 28626 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.103.65:34115 (error 108)
I20260811 14:38:48.758499 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 28966
I20260811 14:38:48.758782 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:38:48.758632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:34115 (local address 127.0.0.1:49998)
0811 14:38:48.758695 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 14:38:48.758743 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.65:34115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 14:38:48.759119 28626 catalog_manager.cc:4584] TS d506e40242e7432bab279e86d17dc3eb (127.2.103.65:34115): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 425395c1e4424c1e8d3ed1c0d495a62c: Network error: Client connection negotiation failed: client connection to 127.2.103.65:34115: connect: Connection refused (error 111)
I20260811 14:38:48.769531 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 29217
I20260811 14:38:48.791159 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 28596
I20260811 14:39:08.804603 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:45907
--webserver_interface=127.2.103.126
--webserver_port=41727
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:45907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:08.891500 29483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:08.891750 29483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:08.891799 29483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:08.893030 29483 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 14:39:08.893095 29483 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 14:39:08.893115 29483 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 14:39:08.893136 29483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:08.893165 29483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:08.893194 29483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:08.893217 29483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:08.893249 29483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:08.894837 29483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:45907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:45907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=41727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:08.895136 29483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:08.895390 29483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:08.900817 29490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:08.900796 29495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:08.900861 29483 server_base.cc:1034] running on GCE node
W20260811 14:39:08.900796 29492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:08.901363 29483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:08.901649 29483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:08.902791 29483 hybrid_clock.cc:648] HybridClock initialized: now 1786459148902780 us; error 29 us; skew 500 ppm
Aug 11 14:39:08 dist-test-slave-twwt krb5kdc[27682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459148, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:08.905499 29483 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:39:08.905972 29483 webserver.cc:460] Webserver started at http://127.2.103.126:41727/ using document root <none> and password file <none>
I20260811 14:39:08.906234 29483 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:08.906283 29483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:08.907779 29483 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:08.908612 29507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:08.908825 29483 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:08.908967 29483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "96284e71ed334f7bb8a68dbf1109c88a"
format_stamp: "Formatted at 2026-08-11 14:38:27 on dist-test-slave-twwt"
server_key: "20e14ef4eeac1e096e227c2edd8450ac"
server_key_iv: "d1a7518123ff67dd508801ed5a238fa2"
server_key_version: "encryptionkey@0"
I20260811 14:39:08.909343 29483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:08.923122 29483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:08.925287 29483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:08.925482 29483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:08.930073 29483 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:45907
I20260811 14:39:08.930116 29596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:45907 every 8 connection(s)
I20260811 14:39:08.930572 29483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:08.930995 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 29483
I20260811 14:39:08.931313 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:34115
--local_ip_for_outbound_sockets=127.2.103.65
--tserver_master_addrs=127.2.103.126:45907
--webserver_port=43887
--webserver_interface=127.2.103.65
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:39:08.932478 29597 sys_catalog.cc:263] Verifying existing consensus state
I20260811 14:39:08.933432 29597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a: Bootstrap starting.
I20260811 14:39:08.945091 29597 log.cc:824] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:08.948434 29597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:39:08.948834 29597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a: Bootstrap complete.
I20260811 14:39:08.952245 29597 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96284e71ed334f7bb8a68dbf1109c88a" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45907 } }
I20260811 14:39:08.952574 29597 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96284e71ed334f7bb8a68dbf1109c88a, State: Initialized, Role: FOLLOWER
I20260811 14:39:08.952766 29597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96284e71ed334f7bb8a68dbf1109c88a" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45907 } }
I20260811 14:39:08.952842 29597 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:08.952883 29597 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:08.952930 29597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 2 FOLLOWER]: Advancing to term 3
I20260811 14:39:08.954316 29597 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96284e71ed334f7bb8a68dbf1109c88a" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45907 } }
I20260811 14:39:08.954438 29597 leader_election.cc:302] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96284e71ed334f7bb8a68dbf1109c88a; no voters:
I20260811 14:39:08.954660 29597 leader_election.cc:288] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 14:39:08.954725 29604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 3 FOLLOWER]: Leader election won for term 3
I20260811 14:39:08.954907 29604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [term 3 LEADER]: Becoming Leader. State: Replica: 96284e71ed334f7bb8a68dbf1109c88a, State: Running, Role: LEADER
I20260811 14:39:08.954967 29597 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:08.955017 29604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96284e71ed334f7bb8a68dbf1109c88a" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45907 } }
I20260811 14:39:08.955334 29605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "96284e71ed334f7bb8a68dbf1109c88a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96284e71ed334f7bb8a68dbf1109c88a" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45907 } } }
I20260811 14:39:08.955417 29605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:08.956043 29614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:08.956370 29606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96284e71ed334f7bb8a68dbf1109c88a. Latest consensus state: current_term: 3 leader_uuid: "96284e71ed334f7bb8a68dbf1109c88a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96284e71ed334f7bb8a68dbf1109c88a" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 45907 } } }
I20260811 14:39:08.956453 29606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:08.956995 29614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:08.957115 29614 catalog_manager.cc:1259] Loaded cluster ID: b488d39b5e4e4f32b50b29b45721f534
I20260811 14:39:08.957154 29614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:39:08.958346 29614 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:08.958554 29614 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a: Loaded TSK: 0
I20260811 14:39:08.958624 29614 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 96284e71ed334f7bb8a68dbf1109c88a: Loaded TSK: 1
F20260811 14:39:08.958881 29614 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f06f43f59ad google::LogMessage::Fail() at ??:0
@ 0x7f06f43f888a google::LogMessage::SendToLog() at ??:0
@ 0x7f06f43f54de google::LogMessage::Flush() at ??:0
@ 0x7f06f43f6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f06f6b62104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f06f6b5fd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f06fc34f1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f06fc34f48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f06fc34f594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f06fc315c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f06fc31fad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f06f62c30d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f06f62b8954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f06f711c6db start_thread at ??:0
@ 0x7f06f2ef361f clone at ??:0
W20260811 14:39:09.065955 29602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:09.066254 29602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:09.066291 29602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:09.068105 29602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:09.068164 29602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:09.068209 29602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:39:09.070533 29602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:34115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=43887
--tserver_master_addrs=127.2.103.126:45907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:09.070851 29602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:09.071138 29602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:09.078523 29669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:09.078926 29671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:09.078953 29668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:09.079414 29602 server_base.cc:1034] running on GCE node
I20260811 14:39:09.079679 29602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:09.080024 29602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:09.081203 29602 hybrid_clock.cc:648] HybridClock initialized: now 1786459149081191 us; error 36 us; skew 500 ppm
Aug 11 14:39:09 dist-test-slave-twwt krb5kdc[27682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459149, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:09.084302 29602 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:39:09.084882 29602 webserver.cc:460] Webserver started at http://127.2.103.65:43887/ using document root <none> and password file <none>
I20260811 14:39:09.085178 29602 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:09.085254 29602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:09.086834 29602 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:39:09.087719 29685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:09.087965 29602 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:39:09.088057 29602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "d506e40242e7432bab279e86d17dc3eb"
format_stamp: "Formatted at 2026-08-11 14:38:27 on dist-test-slave-twwt"
server_key: "0cbb69ca4804f966464e1a39a470d804"
server_key_iv: "d6963dade0cf7805578561cb29c0d495"
server_key_version: "encryptionkey@0"
I20260811 14:39:09.088527 29602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:09.099968 29602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:09.102803 29602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:09.102999 29602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:09.103274 29602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:09.103785 29698 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 14:39:09.106109 29602 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 14:39:09.106195 29602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260811 14:39:09.106278 29602 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 14:39:09.107079 29602 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 14:39:09.107142 29602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:09.107224 29698 tablet_bootstrap.cc:492] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb: Bootstrap starting.
I20260811 14:39:09.109395 29698 log.cc:824] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:09.113207 29698 tablet_bootstrap.cc:492] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:39:09.113663 29698 tablet_bootstrap.cc:492] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb: Bootstrap complete.
I20260811 14:39:09.113860 29698 ts_tablet_manager.cc:1397] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 14:39:09.114391 29602 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:34115
I20260811 14:39:09.114553 29868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:34115 every 8 connection(s)
I20260811 14:39:09.114830 29602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:39:09.116051 29698 raft_consensus.cc:348] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } } peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } }
I20260811 14:39:09.116319 29698 raft_consensus.cc:729] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d506e40242e7432bab279e86d17dc3eb, State: Initialized, Role: FOLLOWER
I20260811 14:39:09.116468 29698 consensus_queue.cc:260] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } } peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } }
I20260811 14:39:09.116756 29698 ts_tablet_manager.cc:1428] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 14:39:09.122763 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 29602
I20260811 14:39:09.123149 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:40427
--local_ip_for_outbound_sockets=127.2.103.66
--tserver_master_addrs=127.2.103.126:45907
--webserver_port=35039
--webserver_interface=127.2.103.66
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:09.214692 29876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:09.214954 29876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:09.214999 29876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:09.216214 29876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:09.216289 29876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:09.216337 29876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
W20260811 14:39:09.217550 29873 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:39:09.115590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.65:50265)
0811 14:39:09.115796 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:09.115855 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:09.116056 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:09.217312 (+101256us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":89,"threads_started":1}
W20260811 14:39:09.217774 29869 heartbeater.cc:644] Failed to heartbeat to 127.2.103.126:45907 (0 consecutive failures): Network error: Failed to ping master at 127.2.103.126:45907: Client connection negotiation failed: client connection to 127.2.103.126:45907: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 14:39:09.218144 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:09.218009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.65:58453)
0811 14:39:09.218070 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:09.218113 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 14:39:09.218124 29876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:40427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=35039
--tserver_master_addrs=127.2.103.126:45907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:09.218412 29876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:09.218658 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:09.218523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.65:59895)
0811 14:39:09.218565 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:09.218598 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260811 14:39:09.218750 29876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:09.218814 29869 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 14:39:09.224265 29885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:09.224274 29884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:09.224283 29890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:09.224541 29876 server_base.cc:1034] running on GCE node
I20260811 14:39:09.224756 29876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:09.225021 29876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:09.226197 29876 hybrid_clock.cc:648] HybridClock initialized: now 1786459149226187 us; error 41 us; skew 500 ppm
Aug 11 14:39:09 dist-test-slave-twwt krb5kdc[27682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459149, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:09.228869 29876 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:39:09.229310 29876 webserver.cc:460] Webserver started at http://127.2.103.66:35039/ using document root <none> and password file <none>
I20260811 14:39:09.229553 29876 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:09.229626 29876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:09.231177 29876 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:39:09.231837 29904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:09.232017 29876 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 14:39:09.232129 29876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "695bbd4ab27d494b85d412e8a37bac91"
format_stamp: "Formatted at 2026-08-11 14:38:27 on dist-test-slave-twwt"
server_key: "53c4f43ebf59d9dc1a021169d9a4eeae"
server_key_iv: "02994e9dc309251ca98f9f2ce500f059"
server_key_version: "encryptionkey@0"
I20260811 14:39:09.232517 29876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:09.241215 29876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:09.243640 29876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:09.243809 29876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:09.244056 29876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:09.244372 29876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:39:09.244428 29876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:09.244498 29876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:39:09.244530 29876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:09.250887 29876 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:40427
I20260811 14:39:09.250994 30113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:40427 every 8 connection(s)
I20260811 14:39:09.251363 29876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:39:09.252503 30116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:09.252108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.66:38427)
0811 14:39:09.252264 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:09.252338 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
W20260811 14:39:09.252985 30114 heartbeater.cc:644] Failed to heartbeat to 127.2.103.126:45907 (0 consecutive failures): Network error: Failed to ping master at 127.2.103.126:45907: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
I20260811 14:39:09.253289 30116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:09.253195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.66:48027)
0811 14:39:09.253243 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:09.253270 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 14:39:09.253693 30116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:09.253620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.66:52871)
0811 14:39:09.253657 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:09.253678 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260811 14:39:09.253808 30114 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 14:39:09.261248 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 29876
I20260811 14:39:09.261571 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:43699
--local_ip_for_outbound_sockets=127.2.103.67
--tserver_master_addrs=127.2.103.126:45907
--webserver_port=38219
--webserver_interface=127.2.103.67
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:09.350700 30120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:09.350966 30120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:09.351013 30120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:09.352290 30120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:09.352375 30120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:09.352423 30120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:39:09.354071 30120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:43699
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=38219
--tserver_master_addrs=127.2.103.126:45907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:09.354372 30120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:09.354681 30120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:09.360288 30129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:09.360283 30126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:09.360306 30120 server_base.cc:1034] running on GCE node
W20260811 14:39:09.360307 30125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:09.360843 30120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:09.361122 30120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:09.362340 30120 hybrid_clock.cc:648] HybridClock initialized: now 1786459149362309 us; error 32 us; skew 500 ppm
Aug 11 14:39:09 dist-test-slave-twwt krb5kdc[27682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459149, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:09.365368 30120 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:39:09.365993 30120 webserver.cc:460] Webserver started at http://127.2.103.67:38219/ using document root <none> and password file <none>
I20260811 14:39:09.366240 30120 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:09.366312 30120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:09.367789 30120 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:09.368480 30144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:09.368664 30120 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:09.368770 30120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8"
format_stamp: "Formatted at 2026-08-11 14:38:27 on dist-test-slave-twwt"
server_key: "e0ce1ca07fa6e32cde6a14c9f26109c8"
server_key_iv: "24405d0de88de0b19ef22f866eb78f8c"
server_key_version: "encryptionkey@0"
I20260811 14:39:09.369154 30120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:09.388996 30120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:09.391588 30120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:09.391775 30120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:09.392041 30120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:09.392393 30120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:39:09.392465 30120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:09.392508 30120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:39:09.392522 30120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:09.398872 30120 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:43699
I20260811 14:39:09.399124 30314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:43699 every 8 connection(s)
I20260811 14:39:09.399369 30120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:39:09.400717 30321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:09.400077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.67:33813)
0811 14:39:09.400233 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:09.400334 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":69,"threads_started":1}
W20260811 14:39:09.401014 30316 heartbeater.cc:644] Failed to heartbeat to 127.2.103.126:45907 (0 consecutive failures): Network error: Failed to ping master at 127.2.103.126:45907: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
I20260811 14:39:09.401341 30321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:09.401248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.67:36637)
0811 14:39:09.401295 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:09.401322 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 14:39:09.401636 30321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:09.401553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.67:41369)
0811 14:39:09.401599 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:09.401621 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 14:39:09.401798 30316 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 14:39:09.408222 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 30120
I20260811 14:39:10.219395 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:10.219162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.65:36725)
0811 14:39:10.219293 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:10.219329 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260811 14:39:10.254395 30325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:10.254179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.66:42299)
0811 14:39:10.254305 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:10.254342 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260811 14:39:10.402436 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:10.402178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.67:37109)
0811 14:39:10.402328 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:10.402370 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260811 14:39:10.590866 30329 raft_consensus.cc:482] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:39:10.591049 30329 raft_consensus.cc:504] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } } peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } }
I20260811 14:39:10.591450 30329 leader_election.cc:288] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 695bbd4ab27d494b85d412e8a37bac91 (127.2.103.66:40427), ac27d7e4e1f3437ba9804c6cd3a5d9d8 (127.2.103.67:43699)
Aug 11 14:39:10 dist-test-slave-twwt krb5kdc[27682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459149, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.66@KRBTEST.COM
Aug 11 14:39:10 dist-test-slave-twwt krb5kdc[27682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459149, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.67@KRBTEST.COM
I20260811 14:39:10.603497 30332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:10.591767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:46813 (local address 127.2.103.67:43699)
0811 14:39:10.591953 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:10.591968 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:10.592015 (+ 47us) server_negotiation.cc:407] Connection header received
0811 14:39:10.592556 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:10.592559 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:10.592601 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:10.592757 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:10.593662 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:10.594734 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:10.595929 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:10.596355 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:10.596487 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:10.599427 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:10.599431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:10.599432 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:10.599445 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:10.601360 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:10.601867 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:10.601870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:10.601870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:10.602012 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:10.602365 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:10.602366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:10.602366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:10.602758 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:10.602851 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:10.603230 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 14:39:10.603377 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":74,"threads_started":1}
I20260811 14:39:10.603503 30334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:10.591821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:41561 (local address 127.2.103.66:40427)
0811 14:39:10.591959 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:10.591974 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:10.592006 (+ 32us) server_negotiation.cc:407] Connection header received
0811 14:39:10.592472 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:10.592475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:10.592531 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:10.592722 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:10.593499 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:10.594681 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:10.595908 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:10.596342 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:10.596491 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:10.599108 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:10.599113 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:10.599113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:10.599126 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:10.601204 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:10.601867 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:10.601870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:10.601870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:10.602010 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:10.602365 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:10.602366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:10.602366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:10.602757 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:10.602851 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:10.603230 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 14:39:10.603377 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
I20260811 14:39:10.603536 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:10.591656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:40427 (local address 127.2.103.65:41561)
0811 14:39:10.591712 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:10.591725 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:10.591793 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:10.592766 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:10.592769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:10.593141 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:39:10.593354 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:10.593360 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:10.594831 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:10.594832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:10.595800 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:10.595803 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:10.596459 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:10.596460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:10.596605 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:10.597238 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:39:10.597248 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:39:10.598976 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:39:10.601311 (+ 2335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:10.601319 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:10.601321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:10.601769 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:10.602092 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:10.602093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:10.602093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:10.602281 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:10.602838 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:39:10.602846 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:39:10.603012 (+ 166us) client_negotiation.cc:769] Sending connection context
0811 14:39:10.603150 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 14:39:10.603360 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":118}
I20260811 14:39:10.603535 30331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:10.591656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:43699 (local address 127.2.103.65:46813)
0811 14:39:10.591865 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:10.591877 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:10.591923 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:10.592861 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:10.592862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:10.593459 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:39:10.593537 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:10.593539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:10.594879 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:10.594880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:10.595807 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:10.595810 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:10.596457 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:10.596459 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:10.596603 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:10.597282 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:39:10.597287 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:39:10.599306 (+ 2019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:39:10.601443 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:10.601444 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:10.601444 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:10.601769 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:10.602094 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:10.602094 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:10.602094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:10.602281 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:10.602838 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:39:10.602846 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:39:10.603012 (+ 166us) client_negotiation.cc:769] Sending connection context
0811 14:39:10.603150 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 14:39:10.603415 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":188,"thread_start_us":67,"threads_started":1}
W20260811 14:39:10.604599 29690 leader_election.cc:341] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 695bbd4ab27d494b85d412e8a37bac91 (127.2.103.66:40427): Not found: Tablet not found: 425395c1e4424c1e8d3ed1c0d495a62c
W20260811 14:39:10.604769 29686 leader_election.cc:341] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ac27d7e4e1f3437ba9804c6cd3a5d9d8 (127.2.103.67:43699): Not found: Tablet not found: 425395c1e4424c1e8d3ed1c0d495a62c
I20260811 14:39:10.604810 29686 leader_election.cc:302] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d506e40242e7432bab279e86d17dc3eb; no voters: 695bbd4ab27d494b85d412e8a37bac91, ac27d7e4e1f3437ba9804c6cd3a5d9d8
I20260811 14:39:10.604929 30329 raft_consensus.cc:2726] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 14:39:11.220345 30335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:11.220010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.65:59737)
0811 14:39:11.220164 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:11.220257 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":75,"threads_started":1}
I20260811 14:39:11.255174 30336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:11.254924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.66:59495)
0811 14:39:11.255092 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:11.255131 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":79,"threads_started":1}
I20260811 14:39:11.403272 30337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:11.403033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.67:38387)
0811 14:39:11.403177 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:11.403219 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":66,"threads_started":1}
I20260811 14:39:12.221287 30338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:12.220962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.65:59203)
0811 14:39:12.221148 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:12.221230 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":114,"threads_started":1}
I20260811 14:39:12.255959 30339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:12.255729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.66:47673)
0811 14:39:12.255861 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:12.255910 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260811 14:39:12.404119 30342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:12.403828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.67:49261)
0811 14:39:12.404009 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:12.404064 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":76,"threads_started":1}
I20260811 14:39:12.530473 30345 raft_consensus.cc:482] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:39:12.530612 30345 raft_consensus.cc:504] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } } peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } }
I20260811 14:39:12.530856 30345 leader_election.cc:288] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 695bbd4ab27d494b85d412e8a37bac91 (127.2.103.66:40427), ac27d7e4e1f3437ba9804c6cd3a5d9d8 (127.2.103.67:43699)
W20260811 14:39:12.531335 29686 leader_election.cc:341] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ac27d7e4e1f3437ba9804c6cd3a5d9d8 (127.2.103.67:43699): Not found: Tablet not found: 425395c1e4424c1e8d3ed1c0d495a62c
W20260811 14:39:12.531417 29690 leader_election.cc:341] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 695bbd4ab27d494b85d412e8a37bac91 (127.2.103.66:40427): Not found: Tablet not found: 425395c1e4424c1e8d3ed1c0d495a62c
I20260811 14:39:12.531541 29686 leader_election.cc:302] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d506e40242e7432bab279e86d17dc3eb; no voters: 695bbd4ab27d494b85d412e8a37bac91, ac27d7e4e1f3437ba9804c6cd3a5d9d8
I20260811 14:39:12.531651 30345 raft_consensus.cc:2726] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 14:39:13.222163 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:13.221889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.65:42607)
0811 14:39:13.222056 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:13.222111 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":82,"threads_started":1}
I20260811 14:39:13.256826 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:13.256573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.66:48107)
0811 14:39:13.256715 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:13.256763 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260811 14:39:13.405061 30349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:13.404751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.67:48943)
0811 14:39:13.404942 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:13.405002 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":98,"threads_started":1}
I20260811 14:39:14.223016 30351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:14.222777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.65:58407)
0811 14:39:14.222923 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:14.222967 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":70,"threads_started":1}
I20260811 14:39:14.257620 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:14.257336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.66:50261)
0811 14:39:14.257526 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:14.257572 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":96,"threads_started":1}
I20260811 14:39:14.405921 30357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:14.405615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.67:48273)
0811 14:39:14.405819 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:14.405869 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":72,"threads_started":1}
I20260811 14:39:14.979167 30358 raft_consensus.cc:482] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:39:14.979307 30358 raft_consensus.cc:504] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } } peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } }
I20260811 14:39:14.979640 30358 leader_election.cc:288] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 695bbd4ab27d494b85d412e8a37bac91 (127.2.103.66:40427), ac27d7e4e1f3437ba9804c6cd3a5d9d8 (127.2.103.67:43699)
W20260811 14:39:14.980096 29690 leader_election.cc:341] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 695bbd4ab27d494b85d412e8a37bac91 (127.2.103.66:40427): Not found: Tablet not found: 425395c1e4424c1e8d3ed1c0d495a62c
W20260811 14:39:14.980212 29686 leader_election.cc:341] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ac27d7e4e1f3437ba9804c6cd3a5d9d8 (127.2.103.67:43699): Not found: Tablet not found: 425395c1e4424c1e8d3ed1c0d495a62c
I20260811 14:39:14.980293 29686 leader_election.cc:302] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d506e40242e7432bab279e86d17dc3eb; no voters: 695bbd4ab27d494b85d412e8a37bac91, ac27d7e4e1f3437ba9804c6cd3a5d9d8
I20260811 14:39:14.980402 30358 raft_consensus.cc:2726] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 14:39:15.223830 30362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:15.223575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.65:37485)
0811 14:39:15.223722 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:15.223767 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
I20260811 14:39:15.258432 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:15.258199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.66:44995)
0811 14:39:15.258337 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:15.258383 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260811 14:39:15.406766 30365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:15.406458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.67:53977)
0811 14:39:15.406664 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:15.406715 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":68,"threads_started":1}
I20260811 14:39:16.224623 30366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:16.224384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.65:33405)
0811 14:39:16.224524 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:16.224574 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260811 14:39:16.259140 30367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:16.258911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.66:44393)
0811 14:39:16.259067 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:16.259098 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260811 14:39:16.407534 30368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:16.407282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.67:51917)
0811 14:39:16.407443 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:16.407483 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":69,"threads_started":1}
I20260811 14:39:17.225487 30372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:17.225193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.65:58709)
0811 14:39:17.225378 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:17.225429 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":89,"threads_started":1}
I20260811 14:39:17.259961 30378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:17.259694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.66:58715)
0811 14:39:17.259876 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:17.259912 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":123,"threads_started":1}
I20260811 14:39:17.408313 30381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:17.408044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.67:50041)
0811 14:39:17.408200 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:17.408245 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":65,"threads_started":1}
I20260811 14:39:18.226321 30383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:18.226052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.65:39265)
0811 14:39:18.226206 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:18.226265 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":67,"threads_started":1}
I20260811 14:39:18.260694 30385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:18.260472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.66:41299)
0811 14:39:18.260603 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:18.260642 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260811 14:39:18.409240 30386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:18.408967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.67:55237)
0811 14:39:18.409137 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:18.409174 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":82,"threads_started":1}
I20260811 14:39:19.227255 30390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:19.226989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.65:51871)
0811 14:39:19.227149 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:19.227198 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":69,"threads_started":1}
I20260811 14:39:19.261590 30391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:19.261317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.66:57495)
0811 14:39:19.261476 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:19.261532 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":84,"threads_started":1}
I20260811 14:39:19.272186 30392 raft_consensus.cc:482] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:39:19.272293 30392 raft_consensus.cc:504] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695bbd4ab27d494b85d412e8a37bac91" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 40427 } } peers { permanent_uuid: "ac27d7e4e1f3437ba9804c6cd3a5d9d8" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 43699 } } peers { permanent_uuid: "d506e40242e7432bab279e86d17dc3eb" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34115 } }
I20260811 14:39:19.272505 30392 leader_election.cc:288] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 695bbd4ab27d494b85d412e8a37bac91 (127.2.103.66:40427), ac27d7e4e1f3437ba9804c6cd3a5d9d8 (127.2.103.67:43699)
W20260811 14:39:19.272953 29686 leader_election.cc:341] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ac27d7e4e1f3437ba9804c6cd3a5d9d8 (127.2.103.67:43699): Not found: Tablet not found: 425395c1e4424c1e8d3ed1c0d495a62c
W20260811 14:39:19.273082 29690 leader_election.cc:341] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 695bbd4ab27d494b85d412e8a37bac91 (127.2.103.66:40427): Not found: Tablet not found: 425395c1e4424c1e8d3ed1c0d495a62c
I20260811 14:39:19.273172 29690 leader_election.cc:302] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d506e40242e7432bab279e86d17dc3eb; no voters: 695bbd4ab27d494b85d412e8a37bac91, ac27d7e4e1f3437ba9804c6cd3a5d9d8
I20260811 14:39:19.273275 30392 raft_consensus.cc:2726] T 425395c1e4424c1e8d3ed1c0d495a62c P d506e40242e7432bab279e86d17dc3eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 14:39:19.409950 30393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:19.409707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.67:57569)
0811 14:39:19.409860 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:19.409896 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260811 14:39:20.227975 30395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:20.227756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.65:37497)
0811 14:39:20.227889 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:20.227924 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260811 14:39:20.262352 30396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:20.262103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.66:49947)
0811 14:39:20.262266 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:20.262305 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":84,"threads_started":1}
I20260811 14:39:20.410863 30397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:20.410540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.67:56147)
0811 14:39:20.410762 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:20.410809 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":132,"threads_started":1}
I20260811 14:39:21.228761 30398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:21.228529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.65:49361)
0811 14:39:21.228668 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:21.228712 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":68,"threads_started":1}
I20260811 14:39:21.263176 30399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:21.262902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.66:34003)
0811 14:39:21.263076 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:21.263127 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":81,"threads_started":1}
I20260811 14:39:21.411635 30400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:21.411398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.67:35493)
0811 14:39:21.411538 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:21.411583 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":68,"threads_started":1}
I20260811 14:39:22.229640 30401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:22.229373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.65:57421)
0811 14:39:22.229541 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:22.229588 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":70,"threads_started":1}
I20260811 14:39:22.263926 30402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:22.263698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.66:60313)
0811 14:39:22.263842 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:22.263878 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":77,"threads_started":1}
I20260811 14:39:22.412412 30403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:22.412153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.67:33361)
0811 14:39:22.412301 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:22.412357 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
I20260811 14:39:23.230472 30404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:23.230233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.65:47195)
0811 14:39:23.230387 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:23.230421 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260811 14:39:23.264743 30405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:23.264466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.66:34153)
0811 14:39:23.264635 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:23.264679 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":72,"threads_started":1}
I20260811 14:39:23.413225 30407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:23.412960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.67:35307)
0811 14:39:23.413131 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:23.413172 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":91,"threads_started":1}
I20260811 14:39:24.231395 30408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:24.231153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.65:48227)
0811 14:39:24.231298 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:24.231346 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260811 14:39:24.265601 30409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:24.265309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.66:49947)
0811 14:39:24.265499 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:24.265536 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":125,"threads_started":1}
I20260811 14:39:24.409263 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 29602
I20260811 14:39:24.421717 30410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:24.420220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:45907 (local address 127.2.103.67:42077)
0811 14:39:24.421612 (+ 1392us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:24.421664 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.126:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1342,"thread_start_us":1252,"threads_started":1}
I20260811 14:39:24.426395 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 29876
I20260811 14:39:24.441023 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 30120
2026-08-11T14:39:24Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58720 ms)
[----------] 30 tests from SecurityITest (245266 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:39:24 dist-test-slave-twwt krb5kdc[30415](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:39:24 dist-test-slave-twwt krb5kdc[30415](info): set up 2 sockets
Aug 11 14:39:24 dist-test-slave-twwt krb5kdc[30415](info): commencing operation
krb5kdc: starting...
W20260811 14:39:25.761417 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:39:25 dist-test-slave-twwt krb5kdc[30415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:39:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:39:25.898224 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:40019
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:34845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:40019
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:25.988209 30434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:25.988567 30434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:25.988679 30434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:25.990198 30434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:25.990257 30434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:25.990280 30434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:25.990302 30434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:25.990325 30434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:25.992084 30434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:34845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:40019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:40019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:25.992419 30434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:25.992715 30434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:25.998026 30445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:25.997993 30443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:25.998003 30451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:25.998163 30434 server_base.cc:1034] running on GCE node
I20260811 14:39:25.998528 30434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:25.998858 30434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:26.000025 30434 hybrid_clock.cc:648] HybridClock initialized: now 1786459166000016 us; error 51 us; skew 500 ppm
Aug 11 14:39:25 dist-test-slave-twwt krb5kdc[30415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459165, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:26.002645 30434 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:39:26.003147 30434 webserver.cc:460] Webserver started at http://127.2.103.126:41917/ using document root <none> and password file <none>
I20260811 14:39:26.003345 30434 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:26.003386 30434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:26.003481 30434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:26.004635 30434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "f2ae6c5ef4c447349d0fb464b1ff6ec0"
format_stamp: "Formatted at 2026-08-11 14:39:25 on dist-test-slave-twwt"
server_key: "b07eb63c28210341cd828aba8fce964e"
server_key_iv: "1b14b099dd095a2a167c8198007a7f7f"
server_key_version: "encryptionkey@0"
I20260811 14:39:26.004968 30434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "f2ae6c5ef4c447349d0fb464b1ff6ec0"
format_stamp: "Formatted at 2026-08-11 14:39:25 on dist-test-slave-twwt"
server_key: "b07eb63c28210341cd828aba8fce964e"
server_key_iv: "1b14b099dd095a2a167c8198007a7f7f"
server_key_version: "encryptionkey@0"
I20260811 14:39:26.006313 30434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:39:26.007124 30459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:26.007308 30434 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:39:26.007447 30434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "f2ae6c5ef4c447349d0fb464b1ff6ec0"
format_stamp: "Formatted at 2026-08-11 14:39:25 on dist-test-slave-twwt"
server_key: "b07eb63c28210341cd828aba8fce964e"
server_key_iv: "1b14b099dd095a2a167c8198007a7f7f"
server_key_version: "encryptionkey@0"
I20260811 14:39:26.007508 30434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:26.017808 30434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:26.019972 30434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:26.020097 30434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:26.024349 30434 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:40019
I20260811 14:39:26.024406 30548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:40019 every 8 connection(s)
I20260811 14:39:26.024791 30434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:26.025728 30549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:26.028345 30549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0: Bootstrap starting.
I20260811 14:39:26.028998 30549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:26.029309 30549 log.cc:824] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:26.030122 30549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0: No bootstrap required, opened a new log
I20260811 14:39:26.031857 30549 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2ae6c5ef4c447349d0fb464b1ff6ec0" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40019 } }
I20260811 14:39:26.032027 30549 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:26.032078 30549 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2ae6c5ef4c447349d0fb464b1ff6ec0, State: Initialized, Role: FOLLOWER
I20260811 14:39:26.032272 30549 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2ae6c5ef4c447349d0fb464b1ff6ec0" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40019 } }
I20260811 14:39:26.032382 30549 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:26.032428 30549 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:26.032480 30549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:26.033048 30549 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2ae6c5ef4c447349d0fb464b1ff6ec0" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40019 } }
I20260811 14:39:26.033175 30549 leader_election.cc:302] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2ae6c5ef4c447349d0fb464b1ff6ec0; no voters:
I20260811 14:39:26.033380 30549 leader_election.cc:288] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:26.033427 30552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:26.033650 30552 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0 [term 1 LEADER]: Becoming Leader. State: Replica: f2ae6c5ef4c447349d0fb464b1ff6ec0, State: Running, Role: LEADER
I20260811 14:39:26.033653 30549 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:26.033759 30552 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2ae6c5ef4c447349d0fb464b1ff6ec0" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40019 } }
I20260811 14:39:26.034175 30554 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2ae6c5ef4c447349d0fb464b1ff6ec0. Latest consensus state: current_term: 1 leader_uuid: "f2ae6c5ef4c447349d0fb464b1ff6ec0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2ae6c5ef4c447349d0fb464b1ff6ec0" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40019 } } }
I20260811 14:39:26.034267 30554 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:26.034327 30553 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2ae6c5ef4c447349d0fb464b1ff6ec0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2ae6c5ef4c447349d0fb464b1ff6ec0" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 40019 } } }
I20260811 14:39:26.034417 30553 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:26.034505 30562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:26.034798 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 30434
I20260811 14:39:26.034915 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:26.035156 30562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:26.035178 2461 external_mini_cluster.cc:1428] Setting key 9a549c16020b296be7a8a090a5e4bc64
I20260811 14:39:26.037128 30562 catalog_manager.cc:1347] Generated new cluster ID: 498b41ff738d44ce89e52bf4e1456dc7
I20260811 14:39:26.037185 30562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:39:26 dist-test-slave-twwt krb5kdc[30415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:39:26.044284 30562 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:26.044984 30562 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:26.047577 30576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.035852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59418 (local address 127.2.103.126:40019)
0811 14:39:26.036251 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:26.036261 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:26.036294 (+ 33us) server_negotiation.cc:407] Connection header received
0811 14:39:26.036430 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:26.036438 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:26.036498 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:26.036666 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:26.037467 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.038402 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.039323 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.040468 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.040580 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:26.044013 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:26.044017 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:26.044017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:26.044031 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:26.045931 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:26.046318 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:26.046322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:26.046322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:26.046426 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:26.046682 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:26.046683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:26.046684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:26.046925 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:26.047101 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:26.047187 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 14:39:26.047355 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":90,"threads_started":1}
I20260811 14:39:26.051693 30562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0: Generated new TSK 0
I20260811 14:39:26.051874 30562 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
I20260811 14:39:26.083261 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:40019
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:34845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:26.170317 30586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:26.170538 30586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:26.170562 30586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:26.171729 30586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:26.171777 30586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:26.171805 30586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:39:26.173365 30586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:34845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:40019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:26.173622 30586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:26.173879 30586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:26.179387 30595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:26.179358 30594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:26.179363 30598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:26.179811 30586 server_base.cc:1034] running on GCE node
I20260811 14:39:26.180117 30586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:26.180399 30586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:26.181555 30586 hybrid_clock.cc:648] HybridClock initialized: now 1786459166181536 us; error 41 us; skew 500 ppm
Aug 11 14:39:26 dist-test-slave-twwt krb5kdc[30415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459166, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:26.184369 30586 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:39:26.184873 30586 webserver.cc:460] Webserver started at http://127.2.103.65:42241/ using document root <none> and password file <none>
I20260811 14:39:26.185117 30586 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:26.185189 30586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:26.185353 30586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:26.186509 30586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "f55ceadae9884e2e913159f707a77159"
format_stamp: "Formatted at 2026-08-11 14:39:26 on dist-test-slave-twwt"
server_key: "c767fb5ba2d43f50d20a586ee3453156"
server_key_iv: "55a5137872d43e2ee08b789d8981e4f8"
server_key_version: "encryptionkey@0"
I20260811 14:39:26.186976 30586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "f55ceadae9884e2e913159f707a77159"
format_stamp: "Formatted at 2026-08-11 14:39:26 on dist-test-slave-twwt"
server_key: "c767fb5ba2d43f50d20a586ee3453156"
server_key_iv: "55a5137872d43e2ee08b789d8981e4f8"
server_key_version: "encryptionkey@0"
I20260811 14:39:26.188419 30586 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:26.189257 30610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:26.189457 30586 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:26.189570 30586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "f55ceadae9884e2e913159f707a77159"
format_stamp: "Formatted at 2026-08-11 14:39:26 on dist-test-slave-twwt"
server_key: "c767fb5ba2d43f50d20a586ee3453156"
server_key_iv: "55a5137872d43e2ee08b789d8981e4f8"
server_key_version: "encryptionkey@0"
I20260811 14:39:26.189648 30586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:26.214994 30586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:26.217612 30586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:26.217798 30586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:26.218075 30586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:26.218423 30586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:39:26.218480 30586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:26.218523 30586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:39:26.218566 30586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:26.224802 30586 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:46503
I20260811 14:39:26.224859 30808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:46503 every 8 connection(s)
I20260811 14:39:26.225196 30586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:39:26.229586 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 30586
I20260811 14:39:26.229738 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:39:26.229962 2461 external_mini_cluster.cc:1428] Setting key ed4dd17188fe157af8207244c96f1b7c
Aug 11 14:39:26 dist-test-slave-twwt krb5kdc[30415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459166, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:39:26.236083 30576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.225935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:48119 (local address 127.2.103.126:40019)
0811 14:39:26.226005 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:26.226008 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:26.226276 (+ 268us) server_negotiation.cc:407] Connection header received
0811 14:39:26.227166 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:26.227168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:26.227199 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:26.227266 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:26.228279 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.228798 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.229862 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.230185 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.230281 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:26.232938 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:26.232940 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:26.232941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:26.232949 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:26.234491 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:26.234996 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:26.234998 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:26.234998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:26.235077 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:26.235386 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:26.235387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:26.235388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:26.235508 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:26.235584 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:26.235887 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 14:39:26.235990 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 14:39:26.236246 30813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.225918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:40019 (local address 127.2.103.65:48119)
0811 14:39:26.226127 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:26.226185 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:26.227055 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:26.227339 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:26.227343 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:26.227959 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:39:26.228163 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:26.228168 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.228888 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.228889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:26.229752 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:26.229755 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.230302 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.230304 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:26.230467 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:26.231072 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:39:26.231080 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:39:26.232838 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:39:26.234574 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:26.234582 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:26.234585 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:26.234908 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:26.235155 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:26.235156 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:26.235156 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:26.235319 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:26.235585 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:39:26.235593 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:39:26.235738 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 14:39:26.235841 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 14:39:26.236009 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":120,"threads_started":1}
I20260811 14:39:26.236754 30809 heartbeater.cc:347] Connected to a master server at 127.2.103.126:40019
I20260811 14:39:26.236876 30809 heartbeater.cc:464] Registering TS with master...
I20260811 14:39:26.237100 30809 heartbeater.cc:511] Master 127.2.103.126:40019 requested a full tablet report, sending...
I20260811 14:39:26.237663 30488 ts_manager.cc:194] Registered new tserver with Master: f55ceadae9884e2e913159f707a77159 (127.2.103.65:46503)
I20260811 14:39:26.238279 30488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:48119
WARNING: no policy specified for kudu/127.2.103.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
I20260811 14:39:26.258698 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:40019
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:34845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:26.344586 30823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:26.344808 30823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:26.344834 30823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:26.346016 30823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:26.346060 30823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:26.346115 30823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:39:26.347705 30823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:34845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:40019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:26.347965 30823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:26.348199 30823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:26.353413 30835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:26.353442 30833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:26.353612 30823 server_base.cc:1034] running on GCE node
W20260811 14:39:26.353442 30832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:26.353921 30823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:26.354213 30823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:26.355381 30823 hybrid_clock.cc:648] HybridClock initialized: now 1786459166355357 us; error 42 us; skew 500 ppm
Aug 11 14:39:26 dist-test-slave-twwt krb5kdc[30415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459166, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:26.357928 30823 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:39:26.358381 30823 webserver.cc:460] Webserver started at http://127.2.103.66:39791/ using document root <none> and password file <none>
I20260811 14:39:26.358683 30823 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:26.358757 30823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:26.358923 30823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:26.359995 30823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "cbd1ec54aceb427887ac2751df57a1b0"
format_stamp: "Formatted at 2026-08-11 14:39:26 on dist-test-slave-twwt"
server_key: "cd2b35b0c0848972a85a14275f8bfb10"
server_key_iv: "5b0688a513eaffb91b49dcc89481e9ad"
server_key_version: "encryptionkey@0"
I20260811 14:39:26.360380 30823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "cbd1ec54aceb427887ac2751df57a1b0"
format_stamp: "Formatted at 2026-08-11 14:39:26 on dist-test-slave-twwt"
server_key: "cd2b35b0c0848972a85a14275f8bfb10"
server_key_iv: "5b0688a513eaffb91b49dcc89481e9ad"
server_key_version: "encryptionkey@0"
I20260811 14:39:26.361771 30823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:39:26.362589 30848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:26.362950 30823 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:26.363052 30823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "cbd1ec54aceb427887ac2751df57a1b0"
format_stamp: "Formatted at 2026-08-11 14:39:26 on dist-test-slave-twwt"
server_key: "cd2b35b0c0848972a85a14275f8bfb10"
server_key_iv: "5b0688a513eaffb91b49dcc89481e9ad"
server_key_version: "encryptionkey@0"
I20260811 14:39:26.363116 30823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:26.383903 30823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:26.386392 30823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:26.386528 30823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:26.386871 30823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:26.387266 30823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:39:26.387307 30823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:26.387333 30823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:39:26.387348 30823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:26.394084 30823 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:45791
I20260811 14:39:26.394178 31032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:45791 every 8 connection(s)
I20260811 14:39:26.394579 30823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
Aug 11 14:39:26 dist-test-slave-twwt krb5kdc[30415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459166, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:39:26.404675 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 30823
I20260811 14:39:26.404838 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:39:26.404826 30576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.395493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:47245 (local address 127.2.103.126:40019)
0811 14:39:26.395560 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:26.395562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:26.395787 (+ 225us) server_negotiation.cc:407] Connection header received
0811 14:39:26.396339 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:26.396340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:26.396353 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:26.396405 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:26.397423 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.397938 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.399047 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.399251 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.399340 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:26.401774 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:26.401776 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:26.401777 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:26.401789 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:26.403281 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:26.403755 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:26.403756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:26.403757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:26.403828 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:26.404108 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:26.404109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:26.404110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:26.404237 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:26.404317 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:26.404631 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 14:39:26.404727 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 14:39:26.405062 31037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.395464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:40019 (local address 127.2.103.66:47245)
0811 14:39:26.395662 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:26.395701 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:26.396223 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:26.396492 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:26.396496 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:26.397105 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:39:26.397305 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:26.397310 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.398044 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.398045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:26.398955 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:26.398958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.399358 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.399359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:26.399487 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:26.400031 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:39:26.400040 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:39:26.401642 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:39:26.403405 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:26.403415 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:26.403418 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:26.403671 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:26.403914 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:26.403914 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:26.403915 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:26.404038 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:26.404320 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:39:26.404328 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:39:26.404489 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 14:39:26.404608 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 14:39:26.404795 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":96,"threads_started":1}
I20260811 14:39:26.405081 2461 external_mini_cluster.cc:1428] Setting key e7011f9aeaaea35882703e0d75a1d13a
I20260811 14:39:26.405721 31034 heartbeater.cc:347] Connected to a master server at 127.2.103.126:40019
I20260811 14:39:26.405846 31034 heartbeater.cc:464] Registering TS with master...
I20260811 14:39:26.406086 31034 heartbeater.cc:511] Master 127.2.103.126:40019 requested a full tablet report, sending...
I20260811 14:39:26.406596 30483 ts_manager.cc:194] Registered new tserver with Master: cbd1ec54aceb427887ac2751df57a1b0 (127.2.103.66:45791)
I20260811 14:39:26.407181 30483 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:47245
WARNING: no policy specified for kudu/127.2.103.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
I20260811 14:39:26.433988 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:40019
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:34845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:26.521915 31042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:26.522166 31042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:26.522203 31042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:26.523438 31042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:26.523512 31042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:26.523559 31042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:39:26.525101 31042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:34845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:40019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:26.525410 31042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:26.525660 31042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 14:39:26.530966 31042 server_base.cc:1034] running on GCE node
W20260811 14:39:26.530932 31051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:26.530916 31053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:26.530907 31049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:26.531289 31042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:26.531553 31042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:26.532735 31042 hybrid_clock.cc:648] HybridClock initialized: now 1786459166532718 us; error 28 us; skew 500 ppm
Aug 11 14:39:26 dist-test-slave-twwt krb5kdc[30415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459166, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:26.536044 31042 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:39:26.536522 31042 webserver.cc:460] Webserver started at http://127.2.103.67:46377/ using document root <none> and password file <none>
I20260811 14:39:26.536774 31042 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:26.536829 31042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:26.536967 31042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:26.538053 31042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "ef51cbd31a584230bc320fbf9680d6b9"
format_stamp: "Formatted at 2026-08-11 14:39:26 on dist-test-slave-twwt"
server_key: "e09af6301ee138866bea6ba1f0fc9e9b"
server_key_iv: "046aba9f180f11d8e353448e228226b5"
server_key_version: "encryptionkey@0"
I20260811 14:39:26.538416 31042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "ef51cbd31a584230bc320fbf9680d6b9"
format_stamp: "Formatted at 2026-08-11 14:39:26 on dist-test-slave-twwt"
server_key: "e09af6301ee138866bea6ba1f0fc9e9b"
server_key_iv: "046aba9f180f11d8e353448e228226b5"
server_key_version: "encryptionkey@0"
I20260811 14:39:26.539851 31042 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:26.540658 31061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:26.540866 31042 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:26.540942 31042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "ef51cbd31a584230bc320fbf9680d6b9"
format_stamp: "Formatted at 2026-08-11 14:39:26 on dist-test-slave-twwt"
server_key: "e09af6301ee138866bea6ba1f0fc9e9b"
server_key_iv: "046aba9f180f11d8e353448e228226b5"
server_key_version: "encryptionkey@0"
I20260811 14:39:26.541038 31042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:26.580202 31042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:26.582473 31042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:26.582664 31042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:26.582942 31042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:26.583261 31042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:39:26.583294 31042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:26.583318 31042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:39:26.583367 31042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:26.589359 31042 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:37641
I20260811 14:39:26.589433 31250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:37641 every 8 connection(s)
I20260811 14:39:26.589766 31042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:39:26.591202 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 31042
I20260811 14:39:26.591300 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:39:26.591519 2461 external_mini_cluster.cc:1428] Setting key cab0dc1a34cb12ac41c0418bdad6b4b1
Aug 11 14:39:26 dist-test-slave-twwt krb5kdc[30415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459166, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:39:26.600013 30576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.590558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:44017 (local address 127.2.103.126:40019)
0811 14:39:26.590656 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:26.590658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:26.590882 (+ 224us) server_negotiation.cc:407] Connection header received
0811 14:39:26.591476 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:26.591477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:26.591492 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:26.591591 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:26.592827 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.593432 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.594428 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.594682 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.594739 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:26.597203 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:26.597205 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:26.597205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:26.597213 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:26.598574 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:26.599108 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:26.599110 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:26.599110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:26.599177 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:26.599436 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:26.599436 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:26.599437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:26.599553 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:26.599612 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:26.599862 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 14:39:26.599942 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 14:39:26.600167 31253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.590532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:40019 (local address 127.2.103.67:44017)
0811 14:39:26.590731 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:26.590775 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:26.591328 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:26.591671 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:26.591676 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:26.592475 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:39:26.592679 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:26.592684 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.593519 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.593520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:26.594341 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:26.594344 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.594766 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.594768 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:26.594912 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:26.595387 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:39:26.595396 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:39:26.597090 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:39:26.598768 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:26.598774 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:26.598776 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:26.599005 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:26.599251 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:26.599252 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:26.599252 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:26.599368 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:26.599617 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:39:26.599622 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:39:26.599743 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 14:39:26.599831 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 14:39:26.599962 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":123,"threads_started":1}
I20260811 14:39:26.600713 31251 heartbeater.cc:347] Connected to a master server at 127.2.103.126:40019
I20260811 14:39:26.600812 31251 heartbeater.cc:464] Registering TS with master...
I20260811 14:39:26.600988 31251 heartbeater.cc:511] Master 127.2.103.126:40019 requested a full tablet report, sending...
I20260811 14:39:26.601356 30483 ts_manager.cc:194] Registered new tserver with Master: ef51cbd31a584230bc320fbf9680d6b9 (127.2.103.67:37641)
I20260811 14:39:26.601701 30483 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:44017
I20260811 14:39:26.603427 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:39:26.611027 30576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.604895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59432 (local address 127.2.103.126:40019)
0811 14:39:26.604934 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:26.604935 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:26.605080 (+ 145us) server_negotiation.cc:407] Connection header received
0811 14:39:26.605105 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:26.605106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:26.605115 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:26.605160 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:26.605663 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.606127 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.606882 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.607059 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.607124 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:26.608217 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:26.608218 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:26.608218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:26.608224 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:26.609709 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:26.610148 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:26.610149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:26.610150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:26.610206 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:26.610387 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:26.610388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:26.610388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:26.610527 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:26.610601 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:26.610872 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 14:39:26.610939 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:39:26.612814 30483 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:39:26.614435 30483 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:39:26.621878 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.617353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:45791 (local address 127.0.0.1:58812)
0811 14:39:26.617724 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:26.617735 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:26.617782 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:26.618196 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:26.618197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:26.618200 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:26.618258 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:26.618260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.619582 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.619584 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:26.620657 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:26.620660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.621535 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.621537 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:26.621619 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:26.621635 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 14:39:26.621721 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 14:39:26.621786 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":98,"thread_start_us":109,"threads_started":1}
I20260811 14:39:26.622182 31269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.617311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:37641 (local address 127.0.0.1:40824)
0811 14:39:26.617544 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:26.617592 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:26.617665 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:26.618025 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:26.618028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:26.618030 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:26.618117 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:26.618119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.619619 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.619620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:26.620980 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:26.620984 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.621754 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.621756 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:26.621823 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:26.621828 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:39:26.621896 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 14:39:26.622059 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":72,"thread_start_us":88,"threads_started":1}
I20260811 14:39:26.622234 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.617507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40824 (local address 127.2.103.67:37641)
0811 14:39:26.617826 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:26.617831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:26.617841 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:39:26.617889 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:26.617892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:26.617911 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:26.618198 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:26.618283 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.619457 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.621091 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.621659 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.621730 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:26.621790 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:26.621908 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 14:39:26.621953 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":89,"threads_started":1}
I20260811 14:39:26.622376 31273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.617797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48602 (local address 127.2.103.65:46503)
0811 14:39:26.617961 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:26.617965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:26.617977 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:39:26.618037 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:26.618040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:26.618057 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:26.618138 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:26.618692 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.619401 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.621080 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.621845 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.621918 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:26.621974 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:26.622235 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 14:39:26.622282 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260811 14:39:26.622542 31272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.617557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:46503 (local address 127.0.0.1:48602)
0811 14:39:26.617872 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:26.617881 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:26.617919 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:26.618223 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:26.618224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:26.618225 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:26.618261 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:26.618262 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.619507 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.619508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:26.620989 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:26.620991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.622019 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.622020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:26.622096 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:26.622100 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:39:26.622151 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 14:39:26.622188 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":113,"thread_start_us":70,"threads_started":1}
I20260811 14:39:26.622991 31270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.617478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58812 (local address 127.2.103.66:45791)
0811 14:39:26.618045 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:26.618049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:26.618058 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:39:26.618096 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:26.618098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:26.618112 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:26.618197 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:26.618325 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.619456 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.620766 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.621436 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.621517 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:26.621577 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:26.622860 (+ 1283us) server_negotiation.cc:299] Negotiation successful
0811 14:39:26.622913 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":93,"threads_started":1}
I20260811 14:39:26.623093 31144 tablet_service.cc:1467] Processing CreateTablet for tablet 62c2e0421fb94c1ca55953d956b9676d (DEFAULT_TABLE table=test-table [id=eade26ee35a044e48ef37548e5d7505d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:39:26.623306 30692 tablet_service.cc:1467] Processing CreateTablet for tablet 62c2e0421fb94c1ca55953d956b9676d (DEFAULT_TABLE table=test-table [id=eade26ee35a044e48ef37548e5d7505d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:39:26.623518 31144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62c2e0421fb94c1ca55953d956b9676d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:26.623627 30922 tablet_service.cc:1467] Processing CreateTablet for tablet 62c2e0421fb94c1ca55953d956b9676d (DEFAULT_TABLE table=test-table [id=eade26ee35a044e48ef37548e5d7505d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:39:26.623729 30692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62c2e0421fb94c1ca55953d956b9676d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:26.623934 30922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62c2e0421fb94c1ca55953d956b9676d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:26.626355 31277 tablet_bootstrap.cc:492] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159: Bootstrap starting.
I20260811 14:39:26.627166 31279 tablet_bootstrap.cc:492] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9: Bootstrap starting.
I20260811 14:39:26.627197 31277 tablet_bootstrap.cc:654] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:26.627928 31279 tablet_bootstrap.cc:654] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:26.627934 31277 log.cc:824] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:26.628545 31279 log.cc:824] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:26.628782 31277 tablet_bootstrap.cc:492] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159: No bootstrap required, opened a new log
I20260811 14:39:26.628865 31277 ts_tablet_manager.cc:1397] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 14:39:26.628924 31278 tablet_bootstrap.cc:492] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0: Bootstrap starting.
I20260811 14:39:26.629585 31278 tablet_bootstrap.cc:654] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:26.630149 31278 log.cc:824] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:26.630771 31279 tablet_bootstrap.cc:492] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9: No bootstrap required, opened a new log
I20260811 14:39:26.630847 31279 ts_tablet_manager.cc:1397] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 14:39:26.631215 31277 raft_consensus.cc:348] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef51cbd31a584230bc320fbf9680d6b9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37641 } } peers { permanent_uuid: "f55ceadae9884e2e913159f707a77159" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46503 } } peers { permanent_uuid: "cbd1ec54aceb427887ac2751df57a1b0" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45791 } }
I20260811 14:39:26.631408 31277 raft_consensus.cc:374] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:26.631443 31277 raft_consensus.cc:729] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f55ceadae9884e2e913159f707a77159, State: Initialized, Role: FOLLOWER
I20260811 14:39:26.631572 31277 consensus_queue.cc:260] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef51cbd31a584230bc320fbf9680d6b9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37641 } } peers { permanent_uuid: "f55ceadae9884e2e913159f707a77159" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46503 } } peers { permanent_uuid: "cbd1ec54aceb427887ac2751df57a1b0" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45791 } }
I20260811 14:39:26.631876 31277 ts_tablet_manager.cc:1428] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:39:26.631918 30809 heartbeater.cc:503] Master 127.2.103.126:40019 was elected leader, sending a full tablet report...
I20260811 14:39:26.633082 31279 raft_consensus.cc:348] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef51cbd31a584230bc320fbf9680d6b9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37641 } } peers { permanent_uuid: "f55ceadae9884e2e913159f707a77159" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46503 } } peers { permanent_uuid: "cbd1ec54aceb427887ac2751df57a1b0" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45791 } }
I20260811 14:39:26.633270 31279 raft_consensus.cc:374] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:26.633342 31279 raft_consensus.cc:729] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef51cbd31a584230bc320fbf9680d6b9, State: Initialized, Role: FOLLOWER
I20260811 14:39:26.633455 31279 consensus_queue.cc:260] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef51cbd31a584230bc320fbf9680d6b9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37641 } } peers { permanent_uuid: "f55ceadae9884e2e913159f707a77159" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46503 } } peers { permanent_uuid: "cbd1ec54aceb427887ac2751df57a1b0" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45791 } }
I20260811 14:39:26.633724 31279 ts_tablet_manager.cc:1428] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:39:26.633749 31251 heartbeater.cc:503] Master 127.2.103.126:40019 was elected leader, sending a full tablet report...
I20260811 14:39:26.634238 31278 tablet_bootstrap.cc:492] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0: No bootstrap required, opened a new log
I20260811 14:39:26.634321 31278 ts_tablet_manager.cc:1397] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 14:39:26.636036 31278 raft_consensus.cc:348] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef51cbd31a584230bc320fbf9680d6b9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37641 } } peers { permanent_uuid: "f55ceadae9884e2e913159f707a77159" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46503 } } peers { permanent_uuid: "cbd1ec54aceb427887ac2751df57a1b0" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45791 } }
I20260811 14:39:26.636224 31278 raft_consensus.cc:374] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:26.636276 31278 raft_consensus.cc:729] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbd1ec54aceb427887ac2751df57a1b0, State: Initialized, Role: FOLLOWER
I20260811 14:39:26.636408 31278 consensus_queue.cc:260] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef51cbd31a584230bc320fbf9680d6b9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37641 } } peers { permanent_uuid: "f55ceadae9884e2e913159f707a77159" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46503 } } peers { permanent_uuid: "cbd1ec54aceb427887ac2751df57a1b0" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45791 } }
I20260811 14:39:26.636695 31278 ts_tablet_manager.cc:1428] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 14:39:26.636735 31034 heartbeater.cc:503] Master 127.2.103.126:40019 was elected leader, sending a full tablet report...
W20260811 14:39:26.645207 31036 tablet.cc:2369] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:39:26.654973 31287 raft_consensus.cc:482] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:39:26.655189 31287 raft_consensus.cc:504] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef51cbd31a584230bc320fbf9680d6b9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37641 } } peers { permanent_uuid: "f55ceadae9884e2e913159f707a77159" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46503 } } peers { permanent_uuid: "cbd1ec54aceb427887ac2751df57a1b0" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45791 } }
I20260811 14:39:26.655687 31287 leader_election.cc:288] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef51cbd31a584230bc320fbf9680d6b9 (127.2.103.67:37641), cbd1ec54aceb427887ac2751df57a1b0 (127.2.103.66:45791)
I20260811 14:39:26.658797 31270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.655942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:58219 (local address 127.2.103.66:45791)
0811 14:39:26.656007 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:26.656009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:26.656282 (+ 273us) server_negotiation.cc:407] Connection header received
0811 14:39:26.656366 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:26.656367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:26.656377 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:26.656448 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:26.656607 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.657079 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.658016 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.658405 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.658493 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:26.658502 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:26.658692 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 14:39:26.658731 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 14:39:26.658882 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.655953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:53219 (local address 127.2.103.67:37641)
0811 14:39:26.656004 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:26.656006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:26.656016 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:39:26.656046 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:26.656047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:26.656057 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:26.656118 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:26.656381 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.656944 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.657977 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.658414 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.658513 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:26.658522 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:26.658755 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 14:39:26.658804 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:39:26.658916 30813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.655853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:37641 (local address 127.2.103.65:53219)
0811 14:39:26.655909 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:26.655922 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:26.655967 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:26.656171 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:26.656172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:26.656175 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:26.656246 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:26.656248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.657049 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.657051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:26.657868 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:26.657871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.658513 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.658514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:26.658657 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:26.658661 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:39:26.658747 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 14:39:26.658800 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 14:39:26.658806 31291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.655858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:45791 (local address 127.2.103.65:58219)
0811 14:39:26.656201 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:26.656210 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:26.656293 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:26.656452 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:26.656452 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:26.656454 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:26.656528 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:26.656530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.657166 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.657168 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:26.657921 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:26.657923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.658491 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.658492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:26.658569 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:26.658572 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:39:26.658680 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 14:39:26.658721 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":162,"thread_start_us":125,"threads_started":1}
I20260811 14:39:26.659276 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62c2e0421fb94c1ca55953d956b9676d" candidate_uuid: "f55ceadae9884e2e913159f707a77159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef51cbd31a584230bc320fbf9680d6b9" is_pre_election: true
I20260811 14:39:26.659475 31172 raft_consensus.cc:2454] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f55ceadae9884e2e913159f707a77159 in term 0.
I20260811 14:39:26.659435 30960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62c2e0421fb94c1ca55953d956b9676d" candidate_uuid: "f55ceadae9884e2e913159f707a77159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbd1ec54aceb427887ac2751df57a1b0" is_pre_election: true
I20260811 14:39:26.659650 30960 raft_consensus.cc:2454] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f55ceadae9884e2e913159f707a77159 in term 0.
I20260811 14:39:26.659731 30612 leader_election.cc:302] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef51cbd31a584230bc320fbf9680d6b9, f55ceadae9884e2e913159f707a77159; no voters:
I20260811 14:39:26.659890 31287 raft_consensus.cc:2781] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:39:26.659960 31287 raft_consensus.cc:482] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:39:26.659978 31287 raft_consensus.cc:3037] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:26.660665 31287 raft_consensus.cc:504] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef51cbd31a584230bc320fbf9680d6b9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37641 } } peers { permanent_uuid: "f55ceadae9884e2e913159f707a77159" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46503 } } peers { permanent_uuid: "cbd1ec54aceb427887ac2751df57a1b0" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45791 } }
I20260811 14:39:26.660808 31287 leader_election.cc:288] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159 [CANDIDATE]: Term 1 election: Requested vote from peers ef51cbd31a584230bc320fbf9680d6b9 (127.2.103.67:37641), cbd1ec54aceb427887ac2751df57a1b0 (127.2.103.66:45791)
I20260811 14:39:26.660976 30960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62c2e0421fb94c1ca55953d956b9676d" candidate_uuid: "f55ceadae9884e2e913159f707a77159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbd1ec54aceb427887ac2751df57a1b0"
I20260811 14:39:26.660986 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62c2e0421fb94c1ca55953d956b9676d" candidate_uuid: "f55ceadae9884e2e913159f707a77159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef51cbd31a584230bc320fbf9680d6b9"
I20260811 14:39:26.661091 30960 raft_consensus.cc:3037] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:26.661087 31172 raft_consensus.cc:3037] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:26.661844 30960 raft_consensus.cc:2454] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f55ceadae9884e2e913159f707a77159 in term 1.
I20260811 14:39:26.661867 31172 raft_consensus.cc:2454] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f55ceadae9884e2e913159f707a77159 in term 1.
I20260811 14:39:26.662060 30611 leader_election.cc:302] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbd1ec54aceb427887ac2751df57a1b0, f55ceadae9884e2e913159f707a77159; no voters:
I20260811 14:39:26.662194 31287 raft_consensus.cc:2781] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:26.662389 31287 raft_consensus.cc:686] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159 [term 1 LEADER]: Becoming Leader. State: Replica: f55ceadae9884e2e913159f707a77159, State: Running, Role: LEADER
I20260811 14:39:26.662511 31287 consensus_queue.cc:237] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef51cbd31a584230bc320fbf9680d6b9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37641 } } peers { permanent_uuid: "f55ceadae9884e2e913159f707a77159" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46503 } } peers { permanent_uuid: "cbd1ec54aceb427887ac2751df57a1b0" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45791 } }
I20260811 14:39:26.663429 30483 catalog_manager.cc:5495] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159 reported cstate change: term changed from 0 to 1, leader changed from <none> to f55ceadae9884e2e913159f707a77159 (127.2.103.65). New cstate: current_term: 1 leader_uuid: "f55ceadae9884e2e913159f707a77159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef51cbd31a584230bc320fbf9680d6b9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f55ceadae9884e2e913159f707a77159" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cbd1ec54aceb427887ac2751df57a1b0" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45791 } health_report { overall_health: UNKNOWN } } }
I20260811 14:39:26.669062 31273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.666651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48604 (local address 127.2.103.65:46503)
0811 14:39:26.666697 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:26.666699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:26.666707 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:39:26.666772 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:26.666772 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:26.666787 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:26.666847 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:39:26.666964 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.667417 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.668264 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.668483 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.668555 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:26.668679 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:39:26.668800 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:39:26.668868 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:26.668902 (+ 34us) server_negotiation.cc:299] Negotiation successful
0811 14:39:26.668920 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:39:26.670926 31172 raft_consensus.cc:1261] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9 [term 1 FOLLOWER]: Refusing update from remote peer f55ceadae9884e2e913159f707a77159: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:39:26.670926 30960 raft_consensus.cc:1261] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0 [term 1 FOLLOWER]: Refusing update from remote peer f55ceadae9884e2e913159f707a77159: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:39:26.671332 31292 consensus_queue.cc:1035] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef51cbd31a584230bc320fbf9680d6b9" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 37641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:39:26.671479 31287 consensus_queue.cc:1035] T 62c2e0421fb94c1ca55953d956b9676d P f55ceadae9884e2e913159f707a77159 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbd1ec54aceb427887ac2751df57a1b0" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 45791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:39:26.674146 31300 mvcc.cc:205] Tried to move back new op lower bound from 7317336746681106432 to 7317336746650423296. Current Snapshot: MvccSnapshot[applied={T|T < 7317336746681106432}]
I20260811 14:39:26.675041 31304 mvcc.cc:205] Tried to move back new op lower bound from 7317336746681106432 to 7317336746650423296. Current Snapshot: MvccSnapshot[applied={T|T < 7317336746681106432}]
I20260811 14:39:26.676685 30488 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:39:26.676770 30488 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:39:26.677490 30488 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f2ae6c5ef4c447349d0fb464b1ff6ec0: Sending DeleteTablet for 3 replicas of tablet 62c2e0421fb94c1ca55953d956b9676d
I20260811 14:39:26.677856 30922 tablet_service.cc:1514] Processing DeleteTablet for tablet 62c2e0421fb94c1ca55953d956b9676d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:39:26 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:58812
I20260811 14:39:26.677866 31144 tablet_service.cc:1514] Processing DeleteTablet for tablet 62c2e0421fb94c1ca55953d956b9676d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:39:26 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:40824
I20260811 14:39:26.678184 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 30586
I20260811 14:39:26.678222 30692 tablet_service.cc:1514] Processing DeleteTablet for tablet 62c2e0421fb94c1ca55953d956b9676d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:39:26 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:48602
I20260811 14:39:26.684477 31316 tablet_replica.cc:330] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9: stopping tablet replica
I20260811 14:39:26.684664 31316 raft_consensus.cc:2229] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:39:26.684840 31316 raft_consensus.cc:2258] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:39:26.685081 31314 tablet_replica.cc:330] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0: stopping tablet replica
I20260811 14:39:26.685225 31314 raft_consensus.cc:2229] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:39:26.685382 31314 raft_consensus.cc:2258] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:39:26.686273 31316 ts_tablet_manager.cc:1905] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:39:26.688669 31316 ts_tablet_manager.cc:1918] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:39:26.688776 31316 log.cc:1196] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/62c2e0421fb94c1ca55953d956b9676d
I20260811 14:39:26.689121 31316 ts_tablet_manager.cc:1939] T 62c2e0421fb94c1ca55953d956b9676d P ef51cbd31a584230bc320fbf9680d6b9: Deleting consensus metadata
I20260811 14:39:26.689648 30463 catalog_manager.cc:4841] TS ef51cbd31a584230bc320fbf9680d6b9 (127.2.103.67:37641): tablet 62c2e0421fb94c1ca55953d956b9676d (table test-table [id=eade26ee35a044e48ef37548e5d7505d]) successfully deleted
I20260811 14:39:26.691278 31314 ts_tablet_manager.cc:1905] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:39:26.692836 31314 ts_tablet_manager.cc:1918] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:39:26.692934 31314 log.cc:1196] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/62c2e0421fb94c1ca55953d956b9676d
I20260811 14:39:26.693218 31314 ts_tablet_manager.cc:1939] T 62c2e0421fb94c1ca55953d956b9676d P cbd1ec54aceb427887ac2751df57a1b0: Deleting consensus metadata
I20260811 14:39:26.693724 30460 catalog_manager.cc:4841] TS cbd1ec54aceb427887ac2751df57a1b0 (127.2.103.66:45791): tablet 62c2e0421fb94c1ca55953d956b9676d (table test-table [id=eade26ee35a044e48ef37548e5d7505d]) successfully deleted
W20260811 14:39:26.696031 30463 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.103.65:46503 (error 108)
I20260811 14:39:26.696233 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 30823
I20260811 14:39:26.699126 31269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.698964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:46503 (local address 127.0.0.1:48606)
0811 14:39:26.699030 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:26.699078 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.65:46503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 14:39:26.699290 30463 catalog_manager.cc:4584] TS f55ceadae9884e2e913159f707a77159 (127.2.103.65:46503): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 62c2e0421fb94c1ca55953d956b9676d: Network error: Client connection negotiation failed: client connection to 127.2.103.65:46503: connect: Connection refused (error 111)
I20260811 14:39:26.707178 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 31042
I20260811 14:39:26.723279 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 30434
2026-08-11T14:39:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2280 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T14:39:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:26Z Disabled control of system clock
I20260811 14:39:26.763872 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:36669
--webserver_interface=127.2.103.126
--webserver_port=0
--builtin_ntp_servers=127.2.103.84:37877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:36669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 14:39:26.847354 31322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:26.847584 31322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:26.847605 31322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:26.848796 31322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:26.848852 31322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:26.848865 31322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:26.848878 31322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:26.848891 31322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:26.850356 31322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:37877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:36669
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:36669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:26.850656 31322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:26.850992 31322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:26.856321 31337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:26.856362 31331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:26.856419 31322 server_base.cc:1034] running on GCE node
W20260811 14:39:26.856321 31335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:26.856818 31322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:26.857141 31322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:26.858349 31322 hybrid_clock.cc:648] HybridClock initialized: now 1786459166858305 us; error 62 us; skew 500 ppm
I20260811 14:39:26.859920 31322 webserver.cc:460] Webserver started at http://127.2.103.126:46225/ using document root <none> and password file <none>
I20260811 14:39:26.860211 31322 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:26.860296 31322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:26.860459 31322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:26.861588 31322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "12d0fda789d94176a6bf828c567f2c66"
format_stamp: "Formatted at 2026-08-11 14:39:26 on dist-test-slave-twwt"
server_key: "1e2f4f8c078032b6e64af135663590c5"
server_key_iv: "59d277e966948429a7441679cee712be"
server_key_version: "encryptionkey@0"
I20260811 14:39:26.861958 31322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "12d0fda789d94176a6bf828c567f2c66"
format_stamp: "Formatted at 2026-08-11 14:39:26 on dist-test-slave-twwt"
server_key: "1e2f4f8c078032b6e64af135663590c5"
server_key_iv: "59d277e966948429a7441679cee712be"
server_key_version: "encryptionkey@0"
I20260811 14:39:26.863446 31322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:39:26.864260 31346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:26.864452 31322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:39:26.864562 31322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "12d0fda789d94176a6bf828c567f2c66"
format_stamp: "Formatted at 2026-08-11 14:39:26 on dist-test-slave-twwt"
server_key: "1e2f4f8c078032b6e64af135663590c5"
server_key_iv: "59d277e966948429a7441679cee712be"
server_key_version: "encryptionkey@0"
I20260811 14:39:26.864641 31322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:26.872638 31322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:26.873010 31322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:26.873174 31322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:26.877655 31322 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:36669
I20260811 14:39:26.877694 31443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:36669 every 8 connection(s)
I20260811 14:39:26.878057 31322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:26.878750 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 31322
I20260811 14:39:26.878868 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:26.879069 2461 external_mini_cluster.cc:1428] Setting key 340565a62daa189ccc60db1f4c1fbaef
I20260811 14:39:26.879377 31444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:26.883145 31444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66: Bootstrap starting.
I20260811 14:39:26.883906 31444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:26.884230 31444 log.cc:824] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:26.885145 31444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66: No bootstrap required, opened a new log
I20260811 14:39:26.885907 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:26.879672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47628 (local address 127.2.103.126:36669)
0811 14:39:26.879990 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:26.879999 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:26.880035 (+ 36us) server_negotiation.cc:407] Connection header received
0811 14:39:26.880744 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:26.880756 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:26.880846 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:26.881014 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:26.881198 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.882588 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.883577 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:26.884231 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:26.884372 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:26.885177 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:26.885182 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:26.885182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:26.885398 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:26.885442 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:26.885456 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:26.885524 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:26.885594 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 14:39:26.885677 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":163,"threads_started":1}
I20260811 14:39:26.887418 31444 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0fda789d94176a6bf828c567f2c66" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36669 } }
I20260811 14:39:26.887562 31444 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:26.887585 31444 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12d0fda789d94176a6bf828c567f2c66, State: Initialized, Role: FOLLOWER
I20260811 14:39:26.887715 31444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0fda789d94176a6bf828c567f2c66" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36669 } }
I20260811 14:39:26.887806 31444 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:26.887830 31444 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:26.887885 31444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:26.888448 31444 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0fda789d94176a6bf828c567f2c66" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36669 } }
I20260811 14:39:26.888538 31444 leader_election.cc:302] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12d0fda789d94176a6bf828c567f2c66; no voters:
I20260811 14:39:26.888705 31444 leader_election.cc:288] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:26.888743 31449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:26.888851 31449 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66 [term 1 LEADER]: Becoming Leader. State: Replica: 12d0fda789d94176a6bf828c567f2c66, State: Running, Role: LEADER
I20260811 14:39:26.889009 31444 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:26.889088 31449 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0fda789d94176a6bf828c567f2c66" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36669 } }
I20260811 14:39:26.889461 31450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12d0fda789d94176a6bf828c567f2c66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0fda789d94176a6bf828c567f2c66" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36669 } } }
I20260811 14:39:26.889480 31452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12d0fda789d94176a6bf828c567f2c66. Latest consensus state: current_term: 1 leader_uuid: "12d0fda789d94176a6bf828c567f2c66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0fda789d94176a6bf828c567f2c66" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36669 } } }
I20260811 14:39:26.889647 31450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:26.889665 31452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:26.890134 31464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:26.890589 31464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:26.892128 31464 catalog_manager.cc:1347] Generated new cluster ID: e03e7909d20a4dacb6e8577d4dc25fa7
I20260811 14:39:26.892184 31464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:39:26.901374 31464 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:26.901911 31464 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:26.906144 31464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66: Generated new TSK 0
I20260811 14:39:26.906343 31464 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:39:26.912647 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:36669
--builtin_ntp_servers=127.2.103.84:37877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 14:39:26.998261 31478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:26.998478 31478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:26.998502 31478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:26.999718 31478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:26.999768 31478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:26.999794 31478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:39:27.001250 31478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:37877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:36669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:27.001529 31478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:27.001888 31478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:27.007858 31483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:27.007895 31484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:27.008034 31486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:27.008098 31478 server_base.cc:1034] running on GCE node
I20260811 14:39:27.008363 31478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:27.008635 31478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:27.009799 31478 hybrid_clock.cc:648] HybridClock initialized: now 1786459167009784 us; error 30 us; skew 500 ppm
I20260811 14:39:27.011234 31478 webserver.cc:460] Webserver started at http://127.2.103.65:42631/ using document root <none> and password file <none>
I20260811 14:39:27.011536 31478 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:27.011584 31478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:27.011716 31478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:27.012847 31478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "c0178085acee4272b667124f37f17f1b"
format_stamp: "Formatted at 2026-08-11 14:39:27 on dist-test-slave-twwt"
server_key: "33d28093e9eb5ddb55f5c7a7d1140ee6"
server_key_iv: "cc7b1a4c956b27602b335a137f1a4074"
server_key_version: "encryptionkey@0"
I20260811 14:39:27.013216 31478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "c0178085acee4272b667124f37f17f1b"
format_stamp: "Formatted at 2026-08-11 14:39:27 on dist-test-slave-twwt"
server_key: "33d28093e9eb5ddb55f5c7a7d1140ee6"
server_key_iv: "cc7b1a4c956b27602b335a137f1a4074"
server_key_version: "encryptionkey@0"
I20260811 14:39:27.014578 31478 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 14:39:27.015439 31493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:27.015591 31478 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:39:27.015707 31478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "c0178085acee4272b667124f37f17f1b"
format_stamp: "Formatted at 2026-08-11 14:39:27 on dist-test-slave-twwt"
server_key: "33d28093e9eb5ddb55f5c7a7d1140ee6"
server_key_iv: "cc7b1a4c956b27602b335a137f1a4074"
server_key_version: "encryptionkey@0"
I20260811 14:39:27.015784 31478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:27.024735 31478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:27.025064 31478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:27.025221 31478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:27.025465 31478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:27.025825 31478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:39:27.025882 31478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:27.025925 31478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:39:27.025954 31478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:27.032445 31478 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:34735
I20260811 14:39:27.032500 31683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:34735 every 8 connection(s)
I20260811 14:39:27.032928 31478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:39:27.038461 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:27.033768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:55805 (local address 127.2.103.126:36669)
0811 14:39:27.033821 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:27.033823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:27.034104 (+ 281us) server_negotiation.cc:407] Connection header received
0811 14:39:27.034582 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:27.034583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:27.034595 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:27.034670 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:27.035415 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.035982 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.037014 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.037207 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.037270 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:27.038191 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:27.038193 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:27.038193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:27.038199 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:27.038215 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:27.038219 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:27.038271 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:27.038367 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 14:39:27.038396 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:39:27.038744 31686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:27.033720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:36669 (local address 127.2.103.65:55805)
0811 14:39:27.033923 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:27.033986 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:27.034484 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:27.034758 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:27.034761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:27.034770 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:39:27.035305 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:27.035310 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.036087 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.036088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:27.036921 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:27.036923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.037287 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.037288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:27.037416 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:27.038040 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:39:27.038040 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:39:27.038072 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:39:27.038098 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:39:27.038279 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:39:27.038283 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:39:27.038286 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:39:27.038352 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 14:39:27.038437 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
I20260811 14:39:27.038906 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 31478
I20260811 14:39:27.039016 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:39:27.039283 2461 external_mini_cluster.cc:1428] Setting key 19f8aab9c3c177f17fdfed8dfb3e24cc
I20260811 14:39:27.039458 31684 heartbeater.cc:347] Connected to a master server at 127.2.103.126:36669
I20260811 14:39:27.039579 31684 heartbeater.cc:464] Registering TS with master...
I20260811 14:39:27.039798 31684 heartbeater.cc:511] Master 127.2.103.126:36669 requested a full tablet report, sending...
I20260811 14:39:27.040360 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:36669
--builtin_ntp_servers=127.2.103.84:37877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 14:39:27.040377 31375 ts_manager.cc:194] Registered new tserver with Master: c0178085acee4272b667124f37f17f1b (127.2.103.65:34735)
I20260811 14:39:27.041013 31375 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.103.65:55805
W20260811 14:39:27.128250 31689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:27.128480 31689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:27.128504 31689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:27.129621 31689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:27.129657 31689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:27.129683 31689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:39:27.131086 31689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:37877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:36669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:27.131305 31689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:27.131572 31689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:27.136765 31706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:27.136750 31704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:27.136734 31708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:27.137053 31689 server_base.cc:1034] running on GCE node
I20260811 14:39:27.137308 31689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:27.137558 31689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:27.138669 31689 hybrid_clock.cc:648] HybridClock initialized: now 1786459167138669 us; error 45 us; skew 500 ppm
I20260811 14:39:27.140064 31689 webserver.cc:460] Webserver started at http://127.2.103.66:44677/ using document root <none> and password file <none>
I20260811 14:39:27.140391 31689 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:27.140453 31689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:27.140604 31689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:27.141714 31689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "dffa03759add4e57bac4f8bb719b7288"
format_stamp: "Formatted at 2026-08-11 14:39:27 on dist-test-slave-twwt"
server_key: "95f53875709644c97bba42b876c515d4"
server_key_iv: "9541fa6cfd07c44911186f17415ecb1a"
server_key_version: "encryptionkey@0"
I20260811 14:39:27.142112 31689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "dffa03759add4e57bac4f8bb719b7288"
format_stamp: "Formatted at 2026-08-11 14:39:27 on dist-test-slave-twwt"
server_key: "95f53875709644c97bba42b876c515d4"
server_key_iv: "9541fa6cfd07c44911186f17415ecb1a"
server_key_version: "encryptionkey@0"
I20260811 14:39:27.143555 31689 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:39:27.144424 31719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:27.144615 31689 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:39:27.144709 31689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "dffa03759add4e57bac4f8bb719b7288"
format_stamp: "Formatted at 2026-08-11 14:39:27 on dist-test-slave-twwt"
server_key: "95f53875709644c97bba42b876c515d4"
server_key_iv: "9541fa6cfd07c44911186f17415ecb1a"
server_key_version: "encryptionkey@0"
I20260811 14:39:27.144793 31689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:27.158528 31689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:27.158974 31689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:27.159137 31689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:27.159397 31689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:27.159751 31689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:39:27.159809 31689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:27.159883 31689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:39:27.159914 31689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:27.165942 31689 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:44115
I20260811 14:39:27.166060 31908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:44115 every 8 connection(s)
I20260811 14:39:27.166422 31689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:39:27.166915 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 31689
I20260811 14:39:27.167045 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:39:27.167284 2461 external_mini_cluster.cc:1428] Setting key bfdf125f5abc6ee3519068925cef3ffe
I20260811 14:39:27.168586 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:36669
--builtin_ntp_servers=127.2.103.84:37877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 14:39:27.172917 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:27.167341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:34461 (local address 127.2.103.126:36669)
0811 14:39:27.167402 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:27.167405 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:27.167738 (+ 333us) server_negotiation.cc:407] Connection header received
0811 14:39:27.168307 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:27.168309 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:27.168322 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:27.168399 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:27.169155 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.169894 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.171375 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.171564 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.171623 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:27.172617 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:27.172618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:27.172619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:27.172624 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:27.172641 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:27.172645 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:27.172699 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:27.172824 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 14:39:27.172852 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:39:27.173228 31911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:27.167376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:36669 (local address 127.2.103.66:34461)
0811 14:39:27.167586 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:27.167633 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:27.168219 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:27.168496 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:27.168501 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:27.168512 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:39:27.169063 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:27.169069 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.169994 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.169994 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:27.171273 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:27.171277 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.171657 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.171658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:27.171824 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:27.172428 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:39:27.172429 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:39:27.172472 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:39:27.172504 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:39:27.172731 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:39:27.172737 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:39:27.172741 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:39:27.172820 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 14:39:27.172943 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
I20260811 14:39:27.173851 31909 heartbeater.cc:347] Connected to a master server at 127.2.103.126:36669
I20260811 14:39:27.173978 31909 heartbeater.cc:464] Registering TS with master...
I20260811 14:39:27.174180 31909 heartbeater.cc:511] Master 127.2.103.126:36669 requested a full tablet report, sending...
I20260811 14:39:27.174676 31375 ts_manager.cc:194] Registered new tserver with Master: dffa03759add4e57bac4f8bb719b7288 (127.2.103.66:44115)
I20260811 14:39:27.175231 31375 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.103.66:34461
W20260811 14:39:27.256567 31912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:27.256779 31912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:27.256803 31912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:27.257899 31912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:27.257932 31912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:27.257957 31912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:39:27.259246 31912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:37877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:36669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:27.259454 31912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:27.259714 31912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:27.264851 31923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:27.264875 31925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:27.264918 31922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:27.265175 31912 server_base.cc:1034] running on GCE node
I20260811 14:39:27.265554 31912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:27.265828 31912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:27.266981 31912 hybrid_clock.cc:648] HybridClock initialized: now 1786459167266963 us; error 30 us; skew 500 ppm
I20260811 14:39:27.268388 31912 webserver.cc:460] Webserver started at http://127.2.103.67:44113/ using document root <none> and password file <none>
I20260811 14:39:27.268666 31912 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:27.268715 31912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:27.268857 31912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:27.270015 31912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "ae57e738996a4362bd60171f5bbdceaf"
format_stamp: "Formatted at 2026-08-11 14:39:27 on dist-test-slave-twwt"
server_key: "f617bf4a696813b9258f4aaa1a295858"
server_key_iv: "6d58f48e2baf508691441cbeaa364da0"
server_key_version: "encryptionkey@0"
I20260811 14:39:27.270399 31912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "ae57e738996a4362bd60171f5bbdceaf"
format_stamp: "Formatted at 2026-08-11 14:39:27 on dist-test-slave-twwt"
server_key: "f617bf4a696813b9258f4aaa1a295858"
server_key_iv: "6d58f48e2baf508691441cbeaa364da0"
server_key_version: "encryptionkey@0"
I20260811 14:39:27.271935 31912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:27.272814 31936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:27.273036 31912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:27.273154 31912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "ae57e738996a4362bd60171f5bbdceaf"
format_stamp: "Formatted at 2026-08-11 14:39:27 on dist-test-slave-twwt"
server_key: "f617bf4a696813b9258f4aaa1a295858"
server_key_iv: "6d58f48e2baf508691441cbeaa364da0"
server_key_version: "encryptionkey@0"
I20260811 14:39:27.273237 31912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:27.292246 31912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:27.292620 31912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:27.292810 31912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:27.293095 31912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:27.293468 31912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:39:27.293534 31912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:27.293569 31912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:39:27.293613 31912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:27.299692 31912 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:39417
I20260811 14:39:27.299768 32138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:39417 every 8 connection(s)
I20260811 14:39:27.300156 31912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:39:27.305612 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:27.301056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:34453 (local address 127.2.103.126:36669)
0811 14:39:27.301135 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:27.301138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:27.301434 (+ 296us) server_negotiation.cc:407] Connection header received
0811 14:39:27.301931 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:27.301933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:27.301945 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:27.302000 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:27.302750 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.303294 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.304288 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.304488 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.304548 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:27.305361 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:27.305362 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:27.305363 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:27.305368 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:27.305383 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:27.305387 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:27.305435 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:27.305523 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 14:39:27.305548 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 14:39:27.305851 32144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:27.301048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:36669 (local address 127.2.103.67:34453)
0811 14:39:27.301278 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:27.301315 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:27.301811 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:27.302067 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:27.302070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:27.302078 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:39:27.302650 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:27.302655 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.303395 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.303396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:27.304199 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:27.304201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.304570 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.304571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:27.304684 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:27.305205 (+ 521us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:39:27.305205 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:39:27.305234 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:39:27.305258 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:39:27.305447 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:39:27.305451 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:39:27.305454 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:39:27.305508 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 14:39:27.305589 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":125,"threads_started":1}
I20260811 14:39:27.306026 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 31912
I20260811 14:39:27.306152 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:39:27.306368 32140 heartbeater.cc:347] Connected to a master server at 127.2.103.126:36669
I20260811 14:39:27.306396 2461 external_mini_cluster.cc:1428] Setting key dc3d9560434239930fa5608030037272
I20260811 14:39:27.306481 32140 heartbeater.cc:464] Registering TS with master...
I20260811 14:39:27.306908 32140 heartbeater.cc:511] Master 127.2.103.126:36669 requested a full tablet report, sending...
I20260811 14:39:27.307375 31374 ts_manager.cc:194] Registered new tserver with Master: ae57e738996a4362bd60171f5bbdceaf (127.2.103.67:39417)
I20260811 14:39:27.307757 31374 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.103.67:34453
I20260811 14:39:27.318552 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:39:27.323073 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:27.320170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47632 (local address 127.2.103.126:36669)
0811 14:39:27.320229 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:27.320231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:27.320344 (+ 113us) server_negotiation.cc:407] Connection header received
0811 14:39:27.320396 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:27.320396 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:27.320406 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:27.320464 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:27.320612 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.321128 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.321961 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.322147 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.322206 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:27.322865 (+ 659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:27.322866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:27.322867 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:27.322871 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:27.322886 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:27.322890 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:27.322933 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:27.322986 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 14:39:27.323008 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 14:39:27.324855 31374 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:39:27.326411 31374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:39:27.333602 32156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:27.329345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53738 (local address 127.2.103.66:44115)
0811 14:39:27.329554 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:27.329559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:27.329757 (+ 198us) server_negotiation.cc:407] Connection header received
0811 14:39:27.329820 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:27.329823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:27.329841 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:27.329917 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:27.330122 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.330935 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.332432 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.333188 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.333251 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:27.333326 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:27.333449 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 14:39:27.333498 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":103,"threads_started":1}
I20260811 14:39:27.333617 32151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:27.328996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:44115 (local address 127.0.0.1:53738)
0811 14:39:27.329619 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:27.329629 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:27.329693 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:27.329993 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:27.329994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:27.329996 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:27.330058 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:27.330060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.331019 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.331021 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:27.332350 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:27.332353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.333252 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.333253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:27.333321 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:27.333334 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 14:39:27.333427 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 14:39:27.333502 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"mutex_wait_us":7,"thread_start_us":82,"threads_started":1}
I20260811 14:39:27.334049 32152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:27.328783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:39417 (local address 127.0.0.1:46474)
0811 14:39:27.329618 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:27.329629 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:27.329693 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:27.330011 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:27.330012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:27.330014 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:27.330079 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:27.330081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.331412 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.331413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:27.332256 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:27.332259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.333355 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.333356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:27.333419 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:27.333423 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:39:27.333477 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 14:39:27.333548 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"mutex_wait_us":19,"thread_start_us":113,"threads_started":1}
I20260811 14:39:27.334296 32153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:27.328907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46474 (local address 127.2.103.67:39417)
0811 14:39:27.329116 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:27.329123 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:27.329800 (+ 677us) server_negotiation.cc:407] Connection header received
0811 14:39:27.329904 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:27.329907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:27.329922 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:27.329996 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:27.330175 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.331315 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.332359 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.333118 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.333196 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:27.333300 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:27.333995 (+ 695us) server_negotiation.cc:299] Negotiation successful
0811 14:39:27.334042 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":91,"threads_started":1}
I20260811 14:39:27.334461 32154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:27.328764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:34735 (local address 127.0.0.1:41754)
0811 14:39:27.329158 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:27.329206 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:27.329282 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:27.329689 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:27.329693 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:27.329696 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:27.329775 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:27.329777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.331236 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.331237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:27.332350 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:27.332353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.334234 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.334235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:27.334299 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:27.334304 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:39:27.334336 (+ 32us) client_negotiation.cc:240] Negotiation successful
0811 14:39:27.334384 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":8,"thread_start_us":69,"threads_started":1}
I20260811 14:39:27.334487 31799 tablet_service.cc:1467] Processing CreateTablet for tablet 0a05109d9da84dfca04f9930af25a6ed (DEFAULT_TABLE table=test-table [id=97f910774d7f4cd99d6493eca49043be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:39:27.334934 32155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:27.329264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41754 (local address 127.2.103.65:34735)
0811 14:39:27.329474 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:27.329479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:27.329489 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:39:27.329534 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:27.329536 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:27.329568 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:27.329870 (+ 302us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:27.329931 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.331143 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.333369 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.334128 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.334660 (+ 532us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:27.334773 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:27.334801 (+ 28us) server_negotiation.cc:299] Negotiation successful
0811 14:39:27.334851 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":131,"threads_started":1}
I20260811 14:39:27.334962 31799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a05109d9da84dfca04f9930af25a6ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:27.335057 32021 tablet_service.cc:1467] Processing CreateTablet for tablet 0a05109d9da84dfca04f9930af25a6ed (DEFAULT_TABLE table=test-table [id=97f910774d7f4cd99d6493eca49043be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:39:27.335433 32021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a05109d9da84dfca04f9930af25a6ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:27.335543 31580 tablet_service.cc:1467] Processing CreateTablet for tablet 0a05109d9da84dfca04f9930af25a6ed (DEFAULT_TABLE table=test-table [id=97f910774d7f4cd99d6493eca49043be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:39:27.335879 31580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a05109d9da84dfca04f9930af25a6ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:27.338330 32157 tablet_bootstrap.cc:492] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288: Bootstrap starting.
I20260811 14:39:27.338884 32158 tablet_bootstrap.cc:492] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf: Bootstrap starting.
I20260811 14:39:27.339103 32157 tablet_bootstrap.cc:654] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:27.339231 32159 tablet_bootstrap.cc:492] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b: Bootstrap starting.
I20260811 14:39:27.339479 32157 log.cc:824] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:27.339810 32158 tablet_bootstrap.cc:654] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:27.339916 32159 tablet_bootstrap.cc:654] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:27.340132 32158 log.cc:824] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:27.340263 32159 log.cc:824] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:27.340396 32157 tablet_bootstrap.cc:492] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288: No bootstrap required, opened a new log
I20260811 14:39:27.340471 32157 ts_tablet_manager.cc:1397] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 14:39:27.340942 32159 tablet_bootstrap.cc:492] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b: No bootstrap required, opened a new log
I20260811 14:39:27.340937 32158 tablet_bootstrap.cc:492] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf: No bootstrap required, opened a new log
I20260811 14:39:27.341037 32159 ts_tablet_manager.cc:1397] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:39:27.341039 32158 ts_tablet_manager.cc:1397] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:39:27.343032 32159 raft_consensus.cc:348] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0178085acee4272b667124f37f17f1b" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34735 } } peers { permanent_uuid: "ae57e738996a4362bd60171f5bbdceaf" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39417 } } peers { permanent_uuid: "dffa03759add4e57bac4f8bb719b7288" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44115 } }
I20260811 14:39:27.343201 32159 raft_consensus.cc:374] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:27.343247 32159 raft_consensus.cc:729] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0178085acee4272b667124f37f17f1b, State: Initialized, Role: FOLLOWER
I20260811 14:39:27.343230 32157 raft_consensus.cc:348] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0178085acee4272b667124f37f17f1b" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34735 } } peers { permanent_uuid: "ae57e738996a4362bd60171f5bbdceaf" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39417 } } peers { permanent_uuid: "dffa03759add4e57bac4f8bb719b7288" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44115 } }
I20260811 14:39:27.343410 32157 raft_consensus.cc:374] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:27.343369 32159 consensus_queue.cc:260] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0178085acee4272b667124f37f17f1b" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34735 } } peers { permanent_uuid: "ae57e738996a4362bd60171f5bbdceaf" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39417 } } peers { permanent_uuid: "dffa03759add4e57bac4f8bb719b7288" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44115 } }
I20260811 14:39:27.343454 32157 raft_consensus.cc:729] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dffa03759add4e57bac4f8bb719b7288, State: Initialized, Role: FOLLOWER
I20260811 14:39:27.343674 32159 ts_tablet_manager.cc:1428] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:39:27.343711 31684 heartbeater.cc:503] Master 127.2.103.126:36669 was elected leader, sending a full tablet report...
I20260811 14:39:27.343669 32158 raft_consensus.cc:348] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0178085acee4272b667124f37f17f1b" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34735 } } peers { permanent_uuid: "ae57e738996a4362bd60171f5bbdceaf" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39417 } } peers { permanent_uuid: "dffa03759add4e57bac4f8bb719b7288" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44115 } }
I20260811 14:39:27.343820 32158 raft_consensus.cc:374] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:27.343868 32158 raft_consensus.cc:729] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae57e738996a4362bd60171f5bbdceaf, State: Initialized, Role: FOLLOWER
I20260811 14:39:27.343596 32157 consensus_queue.cc:260] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0178085acee4272b667124f37f17f1b" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34735 } } peers { permanent_uuid: "ae57e738996a4362bd60171f5bbdceaf" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39417 } } peers { permanent_uuid: "dffa03759add4e57bac4f8bb719b7288" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44115 } }
I20260811 14:39:27.343976 32158 consensus_queue.cc:260] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0178085acee4272b667124f37f17f1b" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34735 } } peers { permanent_uuid: "ae57e738996a4362bd60171f5bbdceaf" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39417 } } peers { permanent_uuid: "dffa03759add4e57bac4f8bb719b7288" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44115 } }
I20260811 14:39:27.344218 32158 ts_tablet_manager.cc:1428] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 14:39:27.344218 32157 ts_tablet_manager.cc:1428] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260811 14:39:27.344274 32140 heartbeater.cc:503] Master 127.2.103.126:36669 was elected leader, sending a full tablet report...
I20260811 14:39:27.344357 31909 heartbeater.cc:503] Master 127.2.103.126:36669 was elected leader, sending a full tablet report...
W20260811 14:39:27.417115 31910 tablet.cc:2369] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:39:27.501494 32163 raft_consensus.cc:482] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:39:27.501643 32163 raft_consensus.cc:504] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0178085acee4272b667124f37f17f1b" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34735 } } peers { permanent_uuid: "ae57e738996a4362bd60171f5bbdceaf" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39417 } } peers { permanent_uuid: "dffa03759add4e57bac4f8bb719b7288" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44115 } }
I20260811 14:39:27.502147 32163 leader_election.cc:288] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae57e738996a4362bd60171f5bbdceaf (127.2.103.67:39417), dffa03759add4e57bac4f8bb719b7288 (127.2.103.66:44115)
I20260811 14:39:27.505131 31686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:27.502376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:39417 (local address 127.2.103.65:59237)
0811 14:39:27.502485 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:27.502497 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:27.502548 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:27.502753 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:27.502755 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:27.502756 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:27.502841 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:27.502843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.503496 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.503498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:27.504271 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:27.504273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.504863 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.504864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:27.504945 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:27.504948 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:39:27.505011 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 14:39:27.505056 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 14:39:27.505131 32153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:27.502475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:59237 (local address 127.2.103.67:39417)
0811 14:39:27.502540 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:27.502542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:27.502555 (+ 13us) server_negotiation.cc:407] Connection header received
0811 14:39:27.502671 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:27.502672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:27.502681 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:27.502748 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:27.502925 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.503407 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.504391 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.504766 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.504863 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:27.504873 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:27.505030 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 14:39:27.505069 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 14:39:27.505259 32156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:27.502501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:53947 (local address 127.2.103.66:44115)
0811 14:39:27.502561 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:27.502563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:27.502796 (+ 233us) server_negotiation.cc:407] Connection header received
0811 14:39:27.502839 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:27.502840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:27.502857 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:27.502926 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:27.503092 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.503551 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.504507 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.504878 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.504968 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:27.504976 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:27.505140 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 14:39:27.505176 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 14:39:27.505456 32167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:27.502437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:44115 (local address 127.2.103.65:53947)
0811 14:39:27.502674 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:27.502683 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:27.502745 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:27.502933 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:27.502934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:27.502936 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:27.503016 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:27.503018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.503639 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.503640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:27.504405 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:27.504407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.504969 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.504970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:27.505052 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:27.505055 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:39:27.505118 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 14:39:27.505159 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":119,"threads_started":1}
I20260811 14:39:27.505491 32053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a05109d9da84dfca04f9930af25a6ed" candidate_uuid: "c0178085acee4272b667124f37f17f1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae57e738996a4362bd60171f5bbdceaf" is_pre_election: true
I20260811 14:39:27.505673 32053 raft_consensus.cc:2454] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0178085acee4272b667124f37f17f1b in term 0.
I20260811 14:39:27.505797 31826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a05109d9da84dfca04f9930af25a6ed" candidate_uuid: "c0178085acee4272b667124f37f17f1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dffa03759add4e57bac4f8bb719b7288" is_pre_election: true
I20260811 14:39:27.505944 31496 leader_election.cc:302] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae57e738996a4362bd60171f5bbdceaf, c0178085acee4272b667124f37f17f1b; no voters:
I20260811 14:39:27.505985 31826 raft_consensus.cc:2454] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0178085acee4272b667124f37f17f1b in term 0.
I20260811 14:39:27.506079 32163 raft_consensus.cc:2781] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:39:27.506175 32163 raft_consensus.cc:482] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:39:27.506212 32163 raft_consensus.cc:3037] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:27.507004 32163 raft_consensus.cc:504] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0178085acee4272b667124f37f17f1b" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34735 } } peers { permanent_uuid: "ae57e738996a4362bd60171f5bbdceaf" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39417 } } peers { permanent_uuid: "dffa03759add4e57bac4f8bb719b7288" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44115 } }
I20260811 14:39:27.507169 32163 leader_election.cc:288] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b [CANDIDATE]: Term 1 election: Requested vote from peers ae57e738996a4362bd60171f5bbdceaf (127.2.103.67:39417), dffa03759add4e57bac4f8bb719b7288 (127.2.103.66:44115)
I20260811 14:39:27.507344 31826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a05109d9da84dfca04f9930af25a6ed" candidate_uuid: "c0178085acee4272b667124f37f17f1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dffa03759add4e57bac4f8bb719b7288"
I20260811 14:39:27.507352 32053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a05109d9da84dfca04f9930af25a6ed" candidate_uuid: "c0178085acee4272b667124f37f17f1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae57e738996a4362bd60171f5bbdceaf"
I20260811 14:39:27.507426 31826 raft_consensus.cc:3037] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:27.507444 32053 raft_consensus.cc:3037] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:27.508214 31826 raft_consensus.cc:2454] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0178085acee4272b667124f37f17f1b in term 1.
I20260811 14:39:27.508221 32053 raft_consensus.cc:2454] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0178085acee4272b667124f37f17f1b in term 1.
I20260811 14:39:27.508415 31496 leader_election.cc:302] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae57e738996a4362bd60171f5bbdceaf, c0178085acee4272b667124f37f17f1b; no voters:
I20260811 14:39:27.508566 32163 raft_consensus.cc:2781] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:27.508746 32163 raft_consensus.cc:686] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b [term 1 LEADER]: Becoming Leader. State: Replica: c0178085acee4272b667124f37f17f1b, State: Running, Role: LEADER
I20260811 14:39:27.508867 32163 consensus_queue.cc:237] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0178085acee4272b667124f37f17f1b" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34735 } } peers { permanent_uuid: "ae57e738996a4362bd60171f5bbdceaf" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39417 } } peers { permanent_uuid: "dffa03759add4e57bac4f8bb719b7288" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44115 } }
I20260811 14:39:27.509743 31375 catalog_manager.cc:5495] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b reported cstate change: term changed from 0 to 1, leader changed from <none> to c0178085acee4272b667124f37f17f1b (127.2.103.65). New cstate: current_term: 1 leader_uuid: "c0178085acee4272b667124f37f17f1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0178085acee4272b667124f37f17f1b" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae57e738996a4362bd60171f5bbdceaf" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dffa03759add4e57bac4f8bb719b7288" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44115 } health_report { overall_health: UNKNOWN } } }
I20260811 14:39:27.516023 32155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:27.513723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41764 (local address 127.2.103.65:34735)
0811 14:39:27.513767 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:27.513770 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:27.513781 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:39:27.513843 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:27.513844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:27.513860 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:27.513921 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:39:27.514020 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.514487 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.515272 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:27.515459 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:27.515521 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:27.515676 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:39:27.515787 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:39:27.515863 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:27.515881 (+ 18us) server_negotiation.cc:299] Negotiation successful
0811 14:39:27.515901 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 14:39:27.517894 32053 raft_consensus.cc:1261] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf [term 1 FOLLOWER]: Refusing update from remote peer c0178085acee4272b667124f37f17f1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:39:27.518234 31826 raft_consensus.cc:1261] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288 [term 1 FOLLOWER]: Refusing update from remote peer c0178085acee4272b667124f37f17f1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:39:27.518267 32168 consensus_queue.cc:1035] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae57e738996a4362bd60171f5bbdceaf" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 39417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:39:27.518543 32168 consensus_queue.cc:1035] T 0a05109d9da84dfca04f9930af25a6ed P c0178085acee4272b667124f37f17f1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dffa03759add4e57bac4f8bb719b7288" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 44115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:39:27.521651 32175 mvcc.cc:205] Tried to move back new op lower bound from 7317336750150275072 to 7317336750116823040. Current Snapshot: MvccSnapshot[applied={T|T < 7317336750150275072}]
I20260811 14:39:27.521785 32173 mvcc.cc:205] Tried to move back new op lower bound from 7317336750150275072 to 7317336750116823040. Current Snapshot: MvccSnapshot[applied={T|T < 7317336750150275072 or (T in {7317336750150275072})}]
I20260811 14:39:27.522089 32177 mvcc.cc:205] Tried to move back new op lower bound from 7317336750150275072 to 7317336750116823040. Current Snapshot: MvccSnapshot[applied={T|T < 7317336750150275072}]
I20260811 14:39:27.523502 31374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:39:27.523602 31374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:39:27.524451 31374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 12d0fda789d94176a6bf828c567f2c66: Sending DeleteTablet for 3 replicas of tablet 0a05109d9da84dfca04f9930af25a6ed
I20260811 14:39:27.524816 31580 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a05109d9da84dfca04f9930af25a6ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:39:27 UTC) from {username='slave'} at 127.0.0.1:41754
I20260811 14:39:27.524866 31799 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a05109d9da84dfca04f9930af25a6ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:39:27 UTC) from {username='slave'} at 127.0.0.1:53738
I20260811 14:39:27.525097 32186 tablet_replica.cc:330] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288: stopping tablet replica
I20260811 14:39:27.525113 32021 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a05109d9da84dfca04f9930af25a6ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:39:27 UTC) from {username='slave'} at 127.0.0.1:46474
I20260811 14:39:27.525231 32186 raft_consensus.cc:2229] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:39:27.525353 32186 raft_consensus.cc:2258] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:39:27.525369 32187 tablet_replica.cc:330] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf: stopping tablet replica
I20260811 14:39:27.525545 32187 raft_consensus.cc:2229] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:39:27.525604 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 31478
I20260811 14:39:27.525736 32187 raft_consensus.cc:2258] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:39:27.528945 32187 ts_tablet_manager.cc:1905] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:39:27.531351 32187 ts_tablet_manager.cc:1918] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:39:27.531456 32187 log.cc:1196] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/0a05109d9da84dfca04f9930af25a6ed
I20260811 14:39:27.531792 32187 ts_tablet_manager.cc:1939] T 0a05109d9da84dfca04f9930af25a6ed P ae57e738996a4362bd60171f5bbdceaf: Deleting consensus metadata
I20260811 14:39:27.532356 31349 catalog_manager.cc:4841] TS ae57e738996a4362bd60171f5bbdceaf (127.2.103.67:39417): tablet 0a05109d9da84dfca04f9930af25a6ed (table test-table [id=97f910774d7f4cd99d6493eca49043be]) successfully deleted
I20260811 14:39:27.536846 32186 ts_tablet_manager.cc:1905] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:39:27.538406 32186 ts_tablet_manager.cc:1918] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:39:27.538515 32186 log.cc:1196] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/0a05109d9da84dfca04f9930af25a6ed
I20260811 14:39:27.538908 32186 ts_tablet_manager.cc:1939] T 0a05109d9da84dfca04f9930af25a6ed P dffa03759add4e57bac4f8bb719b7288: Deleting consensus metadata
I20260811 14:39:27.539423 31350 catalog_manager.cc:4841] TS dffa03759add4e57bac4f8bb719b7288 (127.2.103.66:44115): tablet 0a05109d9da84dfca04f9930af25a6ed (table test-table [id=97f910774d7f4cd99d6493eca49043be]) successfully deleted
W20260811 14:39:27.541473 31350 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.103.65:34735 (error 108)
I20260811 14:39:27.542002 32154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:27.541870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:34735 (local address 127.0.0.1:41766)
0811 14:39:27.541932 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:27.541973 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.65:34735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 14:39:27.542115 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 31689
W20260811 14:39:27.542165 31350 catalog_manager.cc:4584] TS c0178085acee4272b667124f37f17f1b (127.2.103.65:34735): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a05109d9da84dfca04f9930af25a6ed: Network error: Client connection negotiation failed: client connection to 127.2.103.65:34735: connect: Connection refused (error 111)
I20260811 14:39:27.556509 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 31912
I20260811 14:39:27.574791 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 31322
2026-08-11T14:39:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (848 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:39:27 dist-test-slave-twwt krb5kdc[32191](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:39:27 dist-test-slave-twwt krb5kdc[32191](info): set up 2 sockets
Aug 11 14:39:27 dist-test-slave-twwt krb5kdc[32191](info): commencing operation
krb5kdc: starting...
W20260811 14:39:28.907807 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:39:28 dist-test-slave-twwt krb5kdc[32191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:39:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:39:29.045657 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:41423
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:41423
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:29.134117 32218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:29.134371 32218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:29.134397 32218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:29.135711 32218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:29.135771 32218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:29.135784 32218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:29.135797 32218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:29.135810 32218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:29.137466 32218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:41423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:41423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:29.137755 32218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:29.138018 32218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:29.143448 32230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:29.143435 32225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:29.143433 32226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:29.143843 32218 server_base.cc:1034] running on GCE node
I20260811 14:39:29.144167 32218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:29.144548 32218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:29.145741 32218 hybrid_clock.cc:648] HybridClock initialized: now 1786459169145712 us; error 42 us; skew 500 ppm
Aug 11 14:39:29 dist-test-slave-twwt krb5kdc[32191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459169, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:29.149583 32218 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:39:29.150209 32218 webserver.cc:460] Webserver started at http://127.2.103.126:39863/ using document root <none> and password file <none>
I20260811 14:39:29.150506 32218 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:29.150580 32218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:29.150763 32218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:29.152486 32218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "b56839b9918440e294751e1306034db6"
format_stamp: "Formatted at 2026-08-11 14:39:29 on dist-test-slave-twwt"
server_key: "f8743ce17a63c96120c03bfad483248a"
server_key_iv: "c67bb98e895ab2c2a41f261d9446933b"
server_key_version: "encryptionkey@0"
I20260811 14:39:29.153077 32218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "b56839b9918440e294751e1306034db6"
format_stamp: "Formatted at 2026-08-11 14:39:29 on dist-test-slave-twwt"
server_key: "f8743ce17a63c96120c03bfad483248a"
server_key_iv: "c67bb98e895ab2c2a41f261d9446933b"
server_key_version: "encryptionkey@0"
I20260811 14:39:29.156875 32218 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 14:39:29.157934 32246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:29.158860 32218 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:29.158988 32218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "b56839b9918440e294751e1306034db6"
format_stamp: "Formatted at 2026-08-11 14:39:29 on dist-test-slave-twwt"
server_key: "f8743ce17a63c96120c03bfad483248a"
server_key_iv: "c67bb98e895ab2c2a41f261d9446933b"
server_key_version: "encryptionkey@0"
I20260811 14:39:29.159089 32218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:29.176803 32218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:29.181357 32218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:29.183431 32218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:29.188644 32218 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:41423
I20260811 14:39:29.188752 32334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:41423 every 8 connection(s)
I20260811 14:39:29.189111 32218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:29.190162 32335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:29.193643 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 32218
I20260811 14:39:29.193768 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:29.193966 2461 external_mini_cluster.cc:1428] Setting key d25e16cb5049e34b0aea11d0fea90ea0
I20260811 14:39:29.194092 32335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6: Bootstrap starting.
I20260811 14:39:29.195010 32335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:29.195485 32335 log.cc:824] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:29.197335 32335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6: No bootstrap required, opened a new log
I20260811 14:39:29.200739 32335 raft_consensus.cc:348] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56839b9918440e294751e1306034db6" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41423 } }
I20260811 14:39:29.201078 32335 raft_consensus.cc:374] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:29.201205 32335 raft_consensus.cc:729] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b56839b9918440e294751e1306034db6, State: Initialized, Role: FOLLOWER
I20260811 14:39:29.201419 32335 consensus_queue.cc:260] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56839b9918440e294751e1306034db6" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41423 } }
I20260811 14:39:29.201530 32335 raft_consensus.cc:388] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:29.201570 32335 raft_consensus.cc:482] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:29.201618 32335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:29.202533 32335 raft_consensus.cc:504] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56839b9918440e294751e1306034db6" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41423 } }
I20260811 14:39:29.203063 32335 leader_election.cc:302] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b56839b9918440e294751e1306034db6; no voters:
I20260811 14:39:29.203311 32335 leader_election.cc:288] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:29.203423 32345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:29.203613 32335 sys_catalog.cc:564] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:29.203660 32345 raft_consensus.cc:686] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6 [term 1 LEADER]: Becoming Leader. State: Replica: b56839b9918440e294751e1306034db6, State: Running, Role: LEADER
I20260811 14:39:29.203879 32345 consensus_queue.cc:237] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56839b9918440e294751e1306034db6" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41423 } }
I20260811 14:39:29.204355 32346 sys_catalog.cc:455] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b56839b9918440e294751e1306034db6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56839b9918440e294751e1306034db6" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41423 } } }
I20260811 14:39:29.204455 32346 sys_catalog.cc:458] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:29.204624 32347 sys_catalog.cc:455] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b56839b9918440e294751e1306034db6. Latest consensus state: current_term: 1 leader_uuid: "b56839b9918440e294751e1306034db6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56839b9918440e294751e1306034db6" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 41423 } } }
I20260811 14:39:29.204704 32347 sys_catalog.cc:458] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:29.204679 32358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 11 14:39:29 dist-test-slave-twwt krb5kdc[32191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:39:29.205559 32358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:29.207966 32358 catalog_manager.cc:1347] Generated new cluster ID: f6829ee13bdd48bd8da76c1433621337
I20260811 14:39:29.208062 32358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:39:29.210150 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:29.194768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33906 (local address 127.2.103.126:41423)
0811 14:39:29.194980 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:29.194990 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:29.195025 (+ 35us) server_negotiation.cc:407] Connection header received
0811 14:39:29.195704 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:29.195714 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:29.195783 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:29.196005 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:29.196962 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.198408 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.200401 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.201064 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.201176 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:29.205530 (+ 4354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:29.205535 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:29.205535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:29.205550 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:29.207804 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:29.208260 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:29.208264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:29.208265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:29.208387 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:29.208618 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:29.208619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:29.208619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:29.208987 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:29.209180 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:29.209704 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 14:39:29.209883 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":81,"threads_started":1}
I20260811 14:39:29.233466 32358 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:29.234052 32358 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:29.250171 32358 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6: Generated new TSK 0
I20260811 14:39:29.250455 32358 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
I20260811 14:39:29.294564 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41423
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:29.403625 32372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:29.403872 32372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:29.403898 32372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:29.405238 32372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:29.405285 32372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:29.405314 32372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:39:29.406804 32372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:29.407048 32372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:29.407243 32372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:29.412398 32380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:29.412416 32382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:29.412398 32385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:29.412802 32372 server_base.cc:1034] running on GCE node
I20260811 14:39:29.412986 32372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:29.413199 32372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:29.414347 32372 hybrid_clock.cc:648] HybridClock initialized: now 1786459169414334 us; error 28 us; skew 500 ppm
Aug 11 14:39:29 dist-test-slave-twwt krb5kdc[32191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459169, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:29.417057 32372 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:39:29.417528 32372 webserver.cc:460] Webserver started at http://127.2.103.65:37703/ using document root <none> and password file <none>
I20260811 14:39:29.417724 32372 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:29.417764 32372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:29.417858 32372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:29.418977 32372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "1ce6a471c43e4a1dbd8d098e21f717b1"
format_stamp: "Formatted at 2026-08-11 14:39:29 on dist-test-slave-twwt"
server_key: "6eb2a7992ad5bf1493f9a24de86945c6"
server_key_iv: "7511b47eedd8f4e8a0cbdd7d60ccadd2"
server_key_version: "encryptionkey@0"
I20260811 14:39:29.419291 32372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "1ce6a471c43e4a1dbd8d098e21f717b1"
format_stamp: "Formatted at 2026-08-11 14:39:29 on dist-test-slave-twwt"
server_key: "6eb2a7992ad5bf1493f9a24de86945c6"
server_key_iv: "7511b47eedd8f4e8a0cbdd7d60ccadd2"
server_key_version: "encryptionkey@0"
I20260811 14:39:29.420614 32372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:39:29.421334 32394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:29.421602 32372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:39:29.421718 32372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "1ce6a471c43e4a1dbd8d098e21f717b1"
format_stamp: "Formatted at 2026-08-11 14:39:29 on dist-test-slave-twwt"
server_key: "6eb2a7992ad5bf1493f9a24de86945c6"
server_key_iv: "7511b47eedd8f4e8a0cbdd7d60ccadd2"
server_key_version: "encryptionkey@0"
I20260811 14:39:29.421806 32372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:29.445828 32372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:29.447981 32372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:29.448155 32372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:29.448414 32372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:29.448745 32372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:39:29.448803 32372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:29.448845 32372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:39:29.448889 32372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:29.454841 32372 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:34759
I20260811 14:39:29.454900 32592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:34759 every 8 connection(s)
I20260811 14:39:29.455212 32372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:39:29 dist-test-slave-twwt krb5kdc[32191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459169, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:39:29.462281 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 32372
I20260811 14:39:29.462391 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:39:29.462723 2461 external_mini_cluster.cc:1428] Setting key 44988db300ff953eb9d38867c2436fec
I20260811 14:39:29.464933 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:29.456026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:54951 (local address 127.2.103.126:41423)
0811 14:39:29.456100 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:29.456102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:29.456321 (+ 219us) server_negotiation.cc:407] Connection header received
0811 14:39:29.456807 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:29.456808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:29.456818 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:29.456876 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:29.457794 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.458295 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.459411 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.459615 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.459667 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:29.461794 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:29.461796 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:29.461796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:29.461803 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:29.463445 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:29.463921 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:29.463922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:29.463922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:29.463974 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:29.464274 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:29.464274 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:29.464275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:29.464389 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:29.464477 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:29.464795 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 14:39:29.464867 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 14:39:29.465253 32596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:29.455975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:41423 (local address 127.2.103.65:54951)
0811 14:39:29.456177 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:29.456220 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:29.456724 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:29.456931 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:29.456934 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:29.457522 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:39:29.457714 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:29.457719 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.458454 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.458455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:29.459331 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:29.459334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.459689 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.459690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:29.459816 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:29.460275 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:39:29.460285 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:39:29.461672 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:39:29.463506 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:29.463515 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:29.463517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:29.463846 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:29.464038 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:29.464039 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:29.464039 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:29.464193 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:29.464482 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:39:29.464489 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:39:29.464655 (+ 166us) client_negotiation.cc:769] Sending connection context
0811 14:39:29.464774 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 14:39:29.464966 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":100,"threads_started":1}
I20260811 14:39:29.465782 32593 heartbeater.cc:347] Connected to a master server at 127.2.103.126:41423
I20260811 14:39:29.465868 32593 heartbeater.cc:464] Registering TS with master...
I20260811 14:39:29.466078 32593 heartbeater.cc:511] Master 127.2.103.126:41423 requested a full tablet report, sending...
I20260811 14:39:29.466715 32276 ts_manager.cc:194] Registered new tserver with Master: 1ce6a471c43e4a1dbd8d098e21f717b1 (127.2.103.65:34759)
I20260811 14:39:29.467317 32276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:54951
WARNING: no policy specified for kudu/127.2.103.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
I20260811 14:39:29.494292 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41423
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:29.581977 32606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:29.582230 32606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:29.582270 32606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:29.583556 32606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:29.583627 32606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:29.583674 32606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:39:29.585283 32606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:29.585582 32606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:29.585843 32606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:29.591144 32621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:29.591151 32617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:29.591167 32616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:29.591184 32606 server_base.cc:1034] running on GCE node
I20260811 14:39:29.591555 32606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:29.591822 32606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:29.592993 32606 hybrid_clock.cc:648] HybridClock initialized: now 1786459169592979 us; error 55 us; skew 500 ppm
Aug 11 14:39:29 dist-test-slave-twwt krb5kdc[32191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459169, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:29.600898 32606 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:39:29.601408 32606 webserver.cc:460] Webserver started at http://127.2.103.66:46125/ using document root <none> and password file <none>
I20260811 14:39:29.601665 32606 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:29.601737 32606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:29.601894 32606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:29.603175 32606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "412672d2bb654cdc982db22237572304"
format_stamp: "Formatted at 2026-08-11 14:39:29 on dist-test-slave-twwt"
server_key: "62bc81cc8411400cc5b21683a7cc2255"
server_key_iv: "6c8960bdfd239f68c715536e9d480f62"
server_key_version: "encryptionkey@0"
I20260811 14:39:29.603643 32606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "412672d2bb654cdc982db22237572304"
format_stamp: "Formatted at 2026-08-11 14:39:29 on dist-test-slave-twwt"
server_key: "62bc81cc8411400cc5b21683a7cc2255"
server_key_iv: "6c8960bdfd239f68c715536e9d480f62"
server_key_version: "encryptionkey@0"
I20260811 14:39:29.605324 32606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:39:29.606176 32632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:29.606398 32606 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:29.606513 32606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "412672d2bb654cdc982db22237572304"
format_stamp: "Formatted at 2026-08-11 14:39:29 on dist-test-slave-twwt"
server_key: "62bc81cc8411400cc5b21683a7cc2255"
server_key_iv: "6c8960bdfd239f68c715536e9d480f62"
server_key_version: "encryptionkey@0"
I20260811 14:39:29.606591 32606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:29.628827 32606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:29.631429 32606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:29.631616 32606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:29.631878 32606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:29.632228 32606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:39:29.632288 32606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:29.632326 32606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:39:29.632380 32606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:29.638739 32606 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:41739
I20260811 14:39:29.638792 369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:41739 every 8 connection(s)
I20260811 14:39:29.639151 32606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:39:29.640707 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 32606
I20260811 14:39:29.640859 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:39:29.641075 2461 external_mini_cluster.cc:1428] Setting key 4896abe6ae3b6a26ef983ca98de6087f
Aug 11 14:39:29 dist-test-slave-twwt krb5kdc[32191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459169, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:39:29.649650 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:29.639946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:57121 (local address 127.2.103.126:41423)
0811 14:39:29.640019 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:29.640022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:29.640319 (+ 297us) server_negotiation.cc:407] Connection header received
0811 14:39:29.640828 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:29.640830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:29.640843 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:29.640913 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:29.642109 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.642672 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.643689 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.643912 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.643994 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:29.646580 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:29.646582 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:29.646582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:29.646591 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:29.648158 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:29.648606 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:29.648607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:29.648608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:29.648674 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:29.648951 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:29.648952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:29.648952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:29.649079 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:29.649154 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:29.649462 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 14:39:29.649554 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 14:39:29.649858 375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:29.639936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:41423 (local address 127.2.103.66:57121)
0811 14:39:29.640163 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:29.640205 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:29.640703 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:29.640995 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:29.640998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:29.641776 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:39:29.642004 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:29.642010 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.642781 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.642782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:29.643591 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:29.643593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.644002 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.644003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:29.644137 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:29.644722 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:39:29.644733 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:39:29.646467 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:39:29.648267 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:29.648276 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:29.648279 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:29.648522 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:29.648750 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:29.648750 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:29.648751 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:29.648881 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:29.649156 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:39:29.649164 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:39:29.649316 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 14:39:29.649424 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 14:39:29.649596 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for kudu/127.2.103.67@KRBTEST.COM; defaulting to no policy
I20260811 14:39:29.650353 370 heartbeater.cc:347] Connected to a master server at 127.2.103.126:41423
I20260811 14:39:29.650458 370 heartbeater.cc:464] Registering TS with master...
I20260811 14:39:29.650700 370 heartbeater.cc:511] Master 127.2.103.126:41423 requested a full tablet report, sending...
I20260811 14:39:29.651158 32276 ts_manager.cc:194] Registered new tserver with Master: 412672d2bb654cdc982db22237572304 (127.2.103.66:41739)
I20260811 14:39:29.651573 32276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:57121
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
I20260811 14:39:29.670423 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41423
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:29.757459 380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:29.757678 380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:29.757704 380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:29.758912 380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:29.758960 380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:29.758986 380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:39:29.760468 380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:41423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:29.760666 380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:29.760874 380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:29.766253 388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:29.766321 387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:29.766253 392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:29.766599 380 server_base.cc:1034] running on GCE node
I20260811 14:39:29.766846 380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:29.767122 380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:29.768296 380 hybrid_clock.cc:648] HybridClock initialized: now 1786459169768273 us; error 42 us; skew 500 ppm
Aug 11 14:39:29 dist-test-slave-twwt krb5kdc[32191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459169, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:29.771057 380 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:39:29.771584 380 webserver.cc:460] Webserver started at http://127.2.103.67:44761/ using document root <none> and password file <none>
I20260811 14:39:29.771864 380 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:29.771935 380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:29.772083 380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:29.773205 380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "31f8e24175a241f2a7046d8ce5582aac"
format_stamp: "Formatted at 2026-08-11 14:39:29 on dist-test-slave-twwt"
server_key: "82f32cae022ae7587e7437f05c2bf3f8"
server_key_iv: "a630c35a80543a630a8f96d0b660fc25"
server_key_version: "encryptionkey@0"
I20260811 14:39:29.773581 380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "31f8e24175a241f2a7046d8ce5582aac"
format_stamp: "Formatted at 2026-08-11 14:39:29 on dist-test-slave-twwt"
server_key: "82f32cae022ae7587e7437f05c2bf3f8"
server_key_iv: "a630c35a80543a630a8f96d0b660fc25"
server_key_version: "encryptionkey@0"
I20260811 14:39:29.775060 380 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:39:29.775821 406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:29.776033 380 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:39:29.776150 380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "31f8e24175a241f2a7046d8ce5582aac"
format_stamp: "Formatted at 2026-08-11 14:39:29 on dist-test-slave-twwt"
server_key: "82f32cae022ae7587e7437f05c2bf3f8"
server_key_iv: "a630c35a80543a630a8f96d0b660fc25"
server_key_version: "encryptionkey@0"
I20260811 14:39:29.776232 380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:29.799935 380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:29.801962 380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:29.802129 380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:29.802385 380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:29.802759 380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:39:29.802821 380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:29.802866 380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:39:29.802897 380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:29.810050 380 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:36421
I20260811 14:39:29.810127 598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:36421 every 8 connection(s)
I20260811 14:39:29.810498 380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:39:29.816715 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 380
I20260811 14:39:29.816808 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:39:29.817051 2461 external_mini_cluster.cc:1428] Setting key a8d906842800cd72545e1dda7601d9d2
Aug 11 14:39:29 dist-test-slave-twwt krb5kdc[32191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459169, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:39:29.821347 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:29.812527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:38581 (local address 127.2.103.126:41423)
0811 14:39:29.812581 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:29.812582 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:29.812798 (+ 216us) server_negotiation.cc:407] Connection header received
0811 14:39:29.813344 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:29.813345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:29.813358 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:29.813401 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:29.814399 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.815013 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.816037 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.816257 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.816334 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:29.818966 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:29.818968 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:29.818968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:29.818975 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:29.820166 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:29.820536 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:29.820536 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:29.820537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:29.820590 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:29.820803 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:29.820803 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:29.820803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:29.820937 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:29.820993 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:29.821214 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 14:39:29.821279 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 14:39:29.821550 605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:29.812496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:41423 (local address 127.2.103.67:38581)
0811 14:39:29.812666 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:29.812718 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:29.813238 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:29.813483 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:29.813487 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:29.814084 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:39:29.814286 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:29.814291 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.815113 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.815115 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:29.815940 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:29.815942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.816363 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.816365 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:29.816492 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:29.817086 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:39:29.817098 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:39:29.818877 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:39:29.820240 (+ 1363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:29.820246 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:29.820248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:29.820471 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:29.820648 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:29.820649 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:29.820649 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:29.820750 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:29.820996 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:39:29.821001 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:39:29.821120 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 14:39:29.821204 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 14:39:29.821337 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":78,"threads_started":1}
I20260811 14:39:29.822062 602 heartbeater.cc:347] Connected to a master server at 127.2.103.126:41423
I20260811 14:39:29.822165 602 heartbeater.cc:464] Registering TS with master...
I20260811 14:39:29.822325 602 heartbeater.cc:511] Master 127.2.103.126:41423 requested a full tablet report, sending...
I20260811 14:39:29.822747 32276 ts_manager.cc:194] Registered new tserver with Master: 31f8e24175a241f2a7046d8ce5582aac (127.2.103.67:36421)
I20260811 14:39:29.823174 32276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:38581
I20260811 14:39:29.829098 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:39:29.836782 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:29.830438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33912 (local address 127.2.103.126:41423)
0811 14:39:29.830499 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:29.830501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:29.830655 (+ 154us) server_negotiation.cc:407] Connection header received
0811 14:39:29.830753 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:29.830754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:29.830763 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:29.830811 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:29.831473 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.831943 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.832674 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.832889 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.832939 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:29.834272 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:29.834273 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:29.834274 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:29.834280 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:29.835666 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:29.836075 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:29.836076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:29.836076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:29.836130 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:29.836316 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:29.836316 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:29.836317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:29.836442 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:29.836523 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:29.836616 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 14:39:29.836666 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 14:39:29.838498 32276 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:39:29.840126 32276 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:39:29.848925 631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:29.842484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:34759 (local address 127.0.0.1:50170)
0811 14:39:29.843371 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:29.843380 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:29.843430 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:29.843926 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:29.843929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:29.843932 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:29.844004 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:29.844006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.845459 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.845460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:29.847238 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:29.847242 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.848584 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.848585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:29.848675 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:29.848693 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 14:39:29.848766 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 14:39:29.848827 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":867,"mutex_wait_us":3,"thread_start_us":132,"threads_started":1}
I20260811 14:39:29.848935 633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:29.842767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50170 (local address 127.2.103.65:34759)
0811 14:39:29.842977 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:29.842989 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:29.843507 (+ 518us) server_negotiation.cc:407] Connection header received
0811 14:39:29.843610 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:29.843613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:29.843633 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:29.843724 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:29.844081 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.845340 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.847353 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.848274 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.848407 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:29.848488 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:29.848796 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 14:39:29.848846 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":89,"threads_started":1}
I20260811 14:39:29.849615 626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:29.842491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:41739 (local address 127.0.0.1:58888)
0811 14:39:29.843464 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:29.843477 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:29.843539 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:29.844286 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:29.844288 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:29.844290 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:29.844353 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:29.844355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.845794 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.845795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:29.847238 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:29.847242 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.849367 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.849368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:29.849441 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:29.849446 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:39:29.849501 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 14:39:29.849542 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":955,"mutex_wait_us":10,"thread_start_us":112,"threads_started":1}
I20260811 14:39:29.849915 634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:29.842776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58888 (local address 127.2.103.66:41739)
0811 14:39:29.844088 (+ 1312us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:29.844093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:29.844105 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:39:29.844154 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:29.844156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:29.844173 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:29.844439 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:29.844524 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.845661 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.848447 (+ 2786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.849253 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.849346 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:29.849411 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:29.849792 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 14:39:29.849842 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1260,"thread_start_us":104,"threads_started":1}
I20260811 14:39:29.849809 32484 tablet_service.cc:1467] Processing CreateTablet for tablet 2318781dfc4e4f94854ac9cdbae4e970 (DEFAULT_TABLE table=test-table [id=9e846a027d4f4a41877d32241c0402ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:39:29.850243 32484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2318781dfc4e4f94854ac9cdbae4e970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:29.850718 32724 tablet_service.cc:1467] Processing CreateTablet for tablet 2318781dfc4e4f94854ac9cdbae4e970 (DEFAULT_TABLE table=test-table [id=9e846a027d4f4a41877d32241c0402ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:39:29.851153 32724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2318781dfc4e4f94854ac9cdbae4e970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:29.854140 637 tablet_bootstrap.cc:492] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1: Bootstrap starting.
I20260811 14:39:29.854981 637 tablet_bootstrap.cc:654] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:29.854996 638 tablet_bootstrap.cc:492] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304: Bootstrap starting.
I20260811 14:39:29.855683 638 tablet_bootstrap.cc:654] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:29.855755 630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:29.842484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:36421 (local address 127.0.0.1:38422)
0811 14:39:29.843088 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:29.843142 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:29.843215 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:29.844111 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:29.844112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:29.844115 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:29.844148 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:29.844149 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.847151 (+ 3002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.847152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:29.848335 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:29.848339 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.855459 (+ 7120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.855460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:29.855534 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:29.855539 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:39:29.855609 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 14:39:29.855669 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":46,"threads_started":1}
I20260811 14:39:29.855783 636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:29.842766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38422 (local address 127.2.103.67:36421)
0811 14:39:29.843762 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:29.843767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:29.843778 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:39:29.843821 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:29.843823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:29.843841 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:29.843924 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:29.845807 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.847028 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.849457 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.855348 (+ 5891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.855451 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:29.855523 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:29.855638 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 14:39:29.855694 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":955,"thread_start_us":130,"threads_started":1}
I20260811 14:39:29.856056 637 log.cc:824] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:29.856304 638 log.cc:824] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:29.856511 480 tablet_service.cc:1467] Processing CreateTablet for tablet 2318781dfc4e4f94854ac9cdbae4e970 (DEFAULT_TABLE table=test-table [id=9e846a027d4f4a41877d32241c0402ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:39:29.856874 480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2318781dfc4e4f94854ac9cdbae4e970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:29.856918 637 tablet_bootstrap.cc:492] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1: No bootstrap required, opened a new log
I20260811 14:39:29.856978 637 ts_tablet_manager.cc:1397] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 14:39:29.857107 638 tablet_bootstrap.cc:492] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304: No bootstrap required, opened a new log
I20260811 14:39:29.857195 638 ts_tablet_manager.cc:1397] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:39:29.858566 638 raft_consensus.cc:348] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ce6a471c43e4a1dbd8d098e21f717b1" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34759 } } peers { permanent_uuid: "31f8e24175a241f2a7046d8ce5582aac" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36421 } } peers { permanent_uuid: "412672d2bb654cdc982db22237572304" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 41739 } }
I20260811 14:39:29.858819 638 raft_consensus.cc:374] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:29.858871 638 raft_consensus.cc:729] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 412672d2bb654cdc982db22237572304, State: Initialized, Role: FOLLOWER
I20260811 14:39:29.858989 638 consensus_queue.cc:260] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ce6a471c43e4a1dbd8d098e21f717b1" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34759 } } peers { permanent_uuid: "31f8e24175a241f2a7046d8ce5582aac" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36421 } } peers { permanent_uuid: "412672d2bb654cdc982db22237572304" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 41739 } }
I20260811 14:39:29.859309 370 heartbeater.cc:503] Master 127.2.103.126:41423 was elected leader, sending a full tablet report...
I20260811 14:39:29.859328 638 ts_tablet_manager.cc:1428] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:39:29.859571 637 raft_consensus.cc:348] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ce6a471c43e4a1dbd8d098e21f717b1" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34759 } } peers { permanent_uuid: "31f8e24175a241f2a7046d8ce5582aac" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36421 } } peers { permanent_uuid: "412672d2bb654cdc982db22237572304" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 41739 } }
I20260811 14:39:29.859763 637 raft_consensus.cc:374] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:29.859802 637 raft_consensus.cc:729] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ce6a471c43e4a1dbd8d098e21f717b1, State: Initialized, Role: FOLLOWER
I20260811 14:39:29.859929 637 consensus_queue.cc:260] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ce6a471c43e4a1dbd8d098e21f717b1" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34759 } } peers { permanent_uuid: "31f8e24175a241f2a7046d8ce5582aac" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36421 } } peers { permanent_uuid: "412672d2bb654cdc982db22237572304" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 41739 } }
I20260811 14:39:29.860226 637 ts_tablet_manager.cc:1428] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:39:29.860416 32593 heartbeater.cc:503] Master 127.2.103.126:41423 was elected leader, sending a full tablet report...
I20260811 14:39:29.860522 653 tablet_bootstrap.cc:492] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac: Bootstrap starting.
I20260811 14:39:29.861181 653 tablet_bootstrap.cc:654] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:29.861662 653 log.cc:824] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:29.862473 653 tablet_bootstrap.cc:492] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac: No bootstrap required, opened a new log
I20260811 14:39:29.862547 653 ts_tablet_manager.cc:1397] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:39:29.864061 653 raft_consensus.cc:348] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ce6a471c43e4a1dbd8d098e21f717b1" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34759 } } peers { permanent_uuid: "31f8e24175a241f2a7046d8ce5582aac" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36421 } } peers { permanent_uuid: "412672d2bb654cdc982db22237572304" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 41739 } }
I20260811 14:39:29.864220 653 raft_consensus.cc:374] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:29.864269 653 raft_consensus.cc:729] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31f8e24175a241f2a7046d8ce5582aac, State: Initialized, Role: FOLLOWER
I20260811 14:39:29.864398 653 consensus_queue.cc:260] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ce6a471c43e4a1dbd8d098e21f717b1" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34759 } } peers { permanent_uuid: "31f8e24175a241f2a7046d8ce5582aac" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36421 } } peers { permanent_uuid: "412672d2bb654cdc982db22237572304" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 41739 } }
I20260811 14:39:29.864681 653 ts_tablet_manager.cc:1428] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:39:29.864706 602 heartbeater.cc:503] Master 127.2.103.126:41423 was elected leader, sending a full tablet report...
W20260811 14:39:29.889762 372 tablet.cc:2369] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:39:29.956002 32594 tablet.cc:2369] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:39:29.976670 649 raft_consensus.cc:482] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:39:29.976810 649 raft_consensus.cc:504] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ce6a471c43e4a1dbd8d098e21f717b1" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34759 } } peers { permanent_uuid: "31f8e24175a241f2a7046d8ce5582aac" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36421 } } peers { permanent_uuid: "412672d2bb654cdc982db22237572304" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 41739 } }
I20260811 14:39:29.977319 649 leader_election.cc:288] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31f8e24175a241f2a7046d8ce5582aac (127.2.103.67:36421), 412672d2bb654cdc982db22237572304 (127.2.103.66:41739)
I20260811 14:39:29.980698 634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:29.977653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:45903 (local address 127.2.103.66:41739)
0811 14:39:29.977716 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:29.977718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:29.977825 (+ 107us) server_negotiation.cc:407] Connection header received
0811 14:39:29.977919 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:29.977920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:29.977931 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:29.977993 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:29.978167 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.978712 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.979823 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.980331 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.980420 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:29.980430 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:29.980581 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 14:39:29.980621 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 14:39:29.980717 659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:29.977562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:41739 (local address 127.2.103.65:45903)
0811 14:39:29.977744 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:29.977754 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:29.977841 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:29.977997 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:29.977999 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:29.978000 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:29.978086 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:29.978088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.978813 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.978814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:29.979714 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:29.979717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.980420 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.980421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:29.980495 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:29.980499 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:39:29.980575 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 14:39:29.980621 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":106,"threads_started":1}
I20260811 14:39:29.980861 636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:29.977707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:49541 (local address 127.2.103.67:36421)
0811 14:39:29.977762 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:29.977765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:29.977896 (+ 131us) server_negotiation.cc:407] Connection header received
0811 14:39:29.977935 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:29.977936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:29.977945 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:29.978015 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:29.978219 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.978859 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.979959 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.980460 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.980555 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:29.980565 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:29.980741 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 14:39:29.980782 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 14:39:29.980875 661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:29.977559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:36421 (local address 127.2.103.65:49541)
0811 14:39:29.977744 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:29.977757 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:29.977840 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:29.978021 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:29.978022 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:29.978024 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:29.978124 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:29.978126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.978945 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.978947 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:29.979813 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:29.979815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.980568 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.980569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:29.980654 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:29.980657 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:39:29.980732 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 14:39:29.980776 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":104,"threads_started":1}
I20260811 14:39:29.981067 32756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2318781dfc4e4f94854ac9cdbae4e970" candidate_uuid: "1ce6a471c43e4a1dbd8d098e21f717b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412672d2bb654cdc982db22237572304" is_pre_election: true
I20260811 14:39:29.981251 32756 raft_consensus.cc:2454] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ce6a471c43e4a1dbd8d098e21f717b1 in term 0.
I20260811 14:39:29.981199 520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2318781dfc4e4f94854ac9cdbae4e970" candidate_uuid: "1ce6a471c43e4a1dbd8d098e21f717b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31f8e24175a241f2a7046d8ce5582aac" is_pre_election: true
I20260811 14:39:29.981374 520 raft_consensus.cc:2454] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ce6a471c43e4a1dbd8d098e21f717b1 in term 0.
I20260811 14:39:29.981511 32399 leader_election.cc:302] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ce6a471c43e4a1dbd8d098e21f717b1, 412672d2bb654cdc982db22237572304; no voters:
I20260811 14:39:29.981701 649 raft_consensus.cc:2781] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:39:29.981777 649 raft_consensus.cc:482] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:39:29.981809 649 raft_consensus.cc:3037] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:29.982585 649 raft_consensus.cc:504] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ce6a471c43e4a1dbd8d098e21f717b1" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34759 } } peers { permanent_uuid: "31f8e24175a241f2a7046d8ce5582aac" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36421 } } peers { permanent_uuid: "412672d2bb654cdc982db22237572304" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 41739 } }
I20260811 14:39:29.982769 649 leader_election.cc:288] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [CANDIDATE]: Term 1 election: Requested vote from peers 31f8e24175a241f2a7046d8ce5582aac (127.2.103.67:36421), 412672d2bb654cdc982db22237572304 (127.2.103.66:41739)
I20260811 14:39:29.982959 520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2318781dfc4e4f94854ac9cdbae4e970" candidate_uuid: "1ce6a471c43e4a1dbd8d098e21f717b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31f8e24175a241f2a7046d8ce5582aac"
I20260811 14:39:29.982971 32756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2318781dfc4e4f94854ac9cdbae4e970" candidate_uuid: "1ce6a471c43e4a1dbd8d098e21f717b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412672d2bb654cdc982db22237572304"
I20260811 14:39:29.983049 520 raft_consensus.cc:3037] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:29.983074 32756 raft_consensus.cc:3037] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:29.983880 520 raft_consensus.cc:2454] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ce6a471c43e4a1dbd8d098e21f717b1 in term 1.
I20260811 14:39:29.984009 32756 raft_consensus.cc:2454] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ce6a471c43e4a1dbd8d098e21f717b1 in term 1.
I20260811 14:39:29.984092 32401 leader_election.cc:302] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ce6a471c43e4a1dbd8d098e21f717b1, 31f8e24175a241f2a7046d8ce5582aac; no voters:
I20260811 14:39:29.984200 649 raft_consensus.cc:2781] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:29.984375 649 raft_consensus.cc:686] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [term 1 LEADER]: Becoming Leader. State: Replica: 1ce6a471c43e4a1dbd8d098e21f717b1, State: Running, Role: LEADER
I20260811 14:39:29.984490 649 consensus_queue.cc:237] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ce6a471c43e4a1dbd8d098e21f717b1" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34759 } } peers { permanent_uuid: "31f8e24175a241f2a7046d8ce5582aac" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36421 } } peers { permanent_uuid: "412672d2bb654cdc982db22237572304" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 41739 } }
I20260811 14:39:29.985469 32276 catalog_manager.cc:5495] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ce6a471c43e4a1dbd8d098e21f717b1 (127.2.103.65). New cstate: current_term: 1 leader_uuid: "1ce6a471c43e4a1dbd8d098e21f717b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ce6a471c43e4a1dbd8d098e21f717b1" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 34759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31f8e24175a241f2a7046d8ce5582aac" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "412672d2bb654cdc982db22237572304" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 41739 } health_report { overall_health: UNKNOWN } } }
I20260811 14:39:29.995105 633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:29.992664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50184 (local address 127.2.103.65:34759)
0811 14:39:29.992739 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:29.992741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:29.992748 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:39:29.992780 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:29.992781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:29.992798 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:29.992862 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:39:29.992992 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.993450 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.994237 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:29.994418 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:29.994468 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:29.994755 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:39:29.994843 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:39:29.994922 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:29.995037 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 14:39:29.995046 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 14:39:29.997025 520 raft_consensus.cc:1261] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac [term 1 FOLLOWER]: Refusing update from remote peer 1ce6a471c43e4a1dbd8d098e21f717b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:39:29.997025 32756 raft_consensus.cc:1261] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304 [term 1 FOLLOWER]: Refusing update from remote peer 1ce6a471c43e4a1dbd8d098e21f717b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:39:29.997471 649 consensus_queue.cc:1035] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31f8e24175a241f2a7046d8ce5582aac" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:39:29.997627 664 consensus_queue.cc:1035] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "412672d2bb654cdc982db22237572304" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 41739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:39:30.001154 673 mvcc.cc:205] Tried to move back new op lower bound from 7317336760304943104 to 7317336760257241088. Current Snapshot: MvccSnapshot[applied={T|T < 7317336760304943104 or (T in {7317336760304943104})}]
I20260811 14:39:30.002713 32274 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:39:30.002820 32274 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:39:30.003608 32274 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b56839b9918440e294751e1306034db6: Sending DeleteTablet for 3 replicas of tablet 2318781dfc4e4f94854ac9cdbae4e970
I20260811 14:39:30.003980 32724 tablet_service.cc:1514] Processing DeleteTablet for tablet 2318781dfc4e4f94854ac9cdbae4e970 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:39:30 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:58888
I20260811 14:39:30.004014 480 tablet_service.cc:1514] Processing DeleteTablet for tablet 2318781dfc4e4f94854ac9cdbae4e970 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:39:30 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:38422
I20260811 14:39:30.004202 686 tablet_replica.cc:330] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304: stopping tablet replica
I20260811 14:39:30.004236 32484 tablet_service.cc:1514] Processing DeleteTablet for tablet 2318781dfc4e4f94854ac9cdbae4e970 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:39:30 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:50170
I20260811 14:39:30.004316 686 raft_consensus.cc:2229] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:39:30.004421 686 raft_consensus.cc:2258] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:39:30.004592 690 tablet_replica.cc:330] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1: stopping tablet replica
I20260811 14:39:30.004693 690 raft_consensus.cc:2229] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:39:30.004791 690 raft_consensus.cc:2258] T 2318781dfc4e4f94854ac9cdbae4e970 P 1ce6a471c43e4a1dbd8d098e21f717b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:39:30.004851 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 32372
I20260811 14:39:30.004922 687 tablet_replica.cc:330] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac: stopping tablet replica
I20260811 14:39:30.005012 687 raft_consensus.cc:2229] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:39:30.005096 687 raft_consensus.cc:2258] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:39:30.005590 686 ts_tablet_manager.cc:1905] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:39:30.005705 687 ts_tablet_manager.cc:1905] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:39:30.007609 687 ts_tablet_manager.cc:1918] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:39:30.007673 687 log.cc:1196] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/2318781dfc4e4f94854ac9cdbae4e970
I20260811 14:39:30.007786 686 ts_tablet_manager.cc:1918] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:39:30.007864 686 log.cc:1196] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/2318781dfc4e4f94854ac9cdbae4e970
I20260811 14:39:30.007929 687 ts_tablet_manager.cc:1939] T 2318781dfc4e4f94854ac9cdbae4e970 P 31f8e24175a241f2a7046d8ce5582aac: Deleting consensus metadata
I20260811 14:39:30.008184 686 ts_tablet_manager.cc:1939] T 2318781dfc4e4f94854ac9cdbae4e970 P 412672d2bb654cdc982db22237572304: Deleting consensus metadata
I20260811 14:39:30.008301 32253 catalog_manager.cc:4841] TS 31f8e24175a241f2a7046d8ce5582aac (127.2.103.67:36421): tablet 2318781dfc4e4f94854ac9cdbae4e970 (table test-table [id=9e846a027d4f4a41877d32241c0402ba]) successfully deleted
I20260811 14:39:30.008621 32252 catalog_manager.cc:4841] TS 412672d2bb654cdc982db22237572304 (127.2.103.66:41739): tablet 2318781dfc4e4f94854ac9cdbae4e970 (table test-table [id=9e846a027d4f4a41877d32241c0402ba]) successfully deleted
W20260811 14:39:30.018146 32248 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.103.65:34759 (error 108)
I20260811 14:39:30.018823 630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:30.018667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:34759 (local address 127.0.0.1:50200)
0811 14:39:30.018737 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:30.018783 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.65:34759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 14:39:30.018828 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 32606
W20260811 14:39:30.018962 32248 catalog_manager.cc:4584] TS 1ce6a471c43e4a1dbd8d098e21f717b1 (127.2.103.65:34759): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2318781dfc4e4f94854ac9cdbae4e970: Network error: Client connection negotiation failed: client connection to 127.2.103.65:34759: connect: Connection refused (error 111)
I20260811 14:39:30.041124 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 380
I20260811 14:39:30.063351 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 32218
2026-08-11T14:39:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2484 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T14:39:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:30Z Disabled control of system clock
I20260811 14:39:30.096972 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:33161
--webserver_interface=127.2.103.126
--webserver_port=0
--builtin_ntp_servers=127.2.103.84:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:33161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 14:39:30.183759 697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:30.184043 697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:30.184067 697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:30.185246 697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:30.185283 697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:30.185297 697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:30.185308 697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:30.185320 697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:30.186837 697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:33161
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:33161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:30.187116 697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:30.187388 697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:30.192662 709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:30.192713 705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:30.192677 702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:30.192868 697 server_base.cc:1034] running on GCE node
I20260811 14:39:30.193228 697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:30.193598 697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:30.194789 697 hybrid_clock.cc:648] HybridClock initialized: now 1786459170194756 us; error 56 us; skew 500 ppm
I20260811 14:39:30.196164 697 webserver.cc:460] Webserver started at http://127.2.103.126:39927/ using document root <none> and password file <none>
I20260811 14:39:30.196451 697 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:30.196527 697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:30.196682 697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:30.197849 697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "0fc7e469e8e844309a6a50dfab256682"
format_stamp: "Formatted at 2026-08-11 14:39:30 on dist-test-slave-twwt"
server_key: "a5eed48000ddfeda322c17b0d1872690"
server_key_iv: "1e8eb8f9978dc4ceac54fd7628e7b138"
server_key_version: "encryptionkey@0"
I20260811 14:39:30.198254 697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "0fc7e469e8e844309a6a50dfab256682"
format_stamp: "Formatted at 2026-08-11 14:39:30 on dist-test-slave-twwt"
server_key: "a5eed48000ddfeda322c17b0d1872690"
server_key_iv: "1e8eb8f9978dc4ceac54fd7628e7b138"
server_key_version: "encryptionkey@0"
I20260811 14:39:30.199793 697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:39:30.200700 724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:30.200980 697 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:39:30.201108 697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "0fc7e469e8e844309a6a50dfab256682"
format_stamp: "Formatted at 2026-08-11 14:39:30 on dist-test-slave-twwt"
server_key: "a5eed48000ddfeda322c17b0d1872690"
server_key_iv: "1e8eb8f9978dc4ceac54fd7628e7b138"
server_key_version: "encryptionkey@0"
I20260811 14:39:30.201195 697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:30.212997 697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:30.213376 697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:30.213548 697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:30.218107 697 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:33161
I20260811 14:39:30.218161 808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:33161 every 8 connection(s)
I20260811 14:39:30.218573 697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:30.219756 810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:30.222127 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 697
I20260811 14:39:30.222266 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:30.222512 2461 external_mini_cluster.cc:1428] Setting key 8fc4feaa2af7d4f018063d9afbad0cba
I20260811 14:39:30.222867 810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682: Bootstrap starting.
I20260811 14:39:30.223698 810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:30.224052 810 log.cc:824] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:30.224905 810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682: No bootstrap required, opened a new log
I20260811 14:39:30.227133 810 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fc7e469e8e844309a6a50dfab256682" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33161 } }
I20260811 14:39:30.227294 810 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:30.227320 810 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fc7e469e8e844309a6a50dfab256682, State: Initialized, Role: FOLLOWER
I20260811 14:39:30.227504 810 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fc7e469e8e844309a6a50dfab256682" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33161 } }
I20260811 14:39:30.227635 810 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:30.227689 810 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:30.227751 810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:30.228586 810 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fc7e469e8e844309a6a50dfab256682" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33161 } }
I20260811 14:39:30.228618 814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:30.223132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38672 (local address 127.2.103.126:33161)
0811 14:39:30.223404 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:30.223429 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:30.223458 (+ 29us) server_negotiation.cc:407] Connection header received
0811 14:39:30.224010 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:30.224019 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:30.224092 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:30.224290 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:30.224539 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.225512 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.226324 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.226821 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.226963 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:30.227835 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:30.227840 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:30.227840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:30.228045 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:30.228081 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:30.228093 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:30.228155 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:30.228287 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 14:39:30.228357 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":99,"threads_started":1}
I20260811 14:39:30.228749 810 leader_election.cc:302] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fc7e469e8e844309a6a50dfab256682; no voters:
I20260811 14:39:30.228946 810 leader_election.cc:288] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:30.229040 819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:30.229206 819 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682 [term 1 LEADER]: Becoming Leader. State: Replica: 0fc7e469e8e844309a6a50dfab256682, State: Running, Role: LEADER
I20260811 14:39:30.229275 810 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:30.229318 819 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fc7e469e8e844309a6a50dfab256682" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33161 } }
I20260811 14:39:30.229699 819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fc7e469e8e844309a6a50dfab256682. Latest consensus state: current_term: 1 leader_uuid: "0fc7e469e8e844309a6a50dfab256682" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fc7e469e8e844309a6a50dfab256682" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33161 } } }
I20260811 14:39:30.229792 819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:30.229708 820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fc7e469e8e844309a6a50dfab256682" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fc7e469e8e844309a6a50dfab256682" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33161 } } }
I20260811 14:39:30.229880 820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:30.230597 827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:30.231156 827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:30.232878 827 catalog_manager.cc:1347] Generated new cluster ID: 83847d8f3e4d409aa81f7ae40312604b
I20260811 14:39:30.232932 827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:39:30.241115 827 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:30.241955 827 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:30.255359 827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682: Generated new TSK 0
I20260811 14:39:30.255620 827 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:39:30.263062 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:33161
--builtin_ntp_servers=127.2.103.84:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 14:39:30.352451 855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:30.352763 855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:30.352807 855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:30.354079 855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:30.354161 855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:30.354208 855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:39:30.355762 855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:33161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:30.356079 855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:30.356439 855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:30.362056 862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:30.362082 864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:30.362140 868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:30.362529 855 server_base.cc:1034] running on GCE node
I20260811 14:39:30.362794 855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:30.363111 855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:30.364289 855 hybrid_clock.cc:648] HybridClock initialized: now 1786459170364267 us; error 38 us; skew 500 ppm
I20260811 14:39:30.365726 855 webserver.cc:460] Webserver started at http://127.2.103.65:34061/ using document root <none> and password file <none>
I20260811 14:39:30.366014 855 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:30.366089 855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:30.366261 855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:30.367496 855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "5291b42aec4f4f29a5cc15f25a957f9c"
format_stamp: "Formatted at 2026-08-11 14:39:30 on dist-test-slave-twwt"
server_key: "7c1e0f4eaa1822581c2a7a85b97d918d"
server_key_iv: "a3aa152b0a9c585bcf78e8d6036efd09"
server_key_version: "encryptionkey@0"
I20260811 14:39:30.367893 855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "5291b42aec4f4f29a5cc15f25a957f9c"
format_stamp: "Formatted at 2026-08-11 14:39:30 on dist-test-slave-twwt"
server_key: "7c1e0f4eaa1822581c2a7a85b97d918d"
server_key_iv: "a3aa152b0a9c585bcf78e8d6036efd09"
server_key_version: "encryptionkey@0"
I20260811 14:39:30.369354 855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:39:30.370277 877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:30.370466 855 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:30.370573 855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "5291b42aec4f4f29a5cc15f25a957f9c"
format_stamp: "Formatted at 2026-08-11 14:39:30 on dist-test-slave-twwt"
server_key: "7c1e0f4eaa1822581c2a7a85b97d918d"
server_key_iv: "a3aa152b0a9c585bcf78e8d6036efd09"
server_key_version: "encryptionkey@0"
I20260811 14:39:30.370692 855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:30.391615 855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:30.391984 855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:30.392150 855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:30.392421 855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:30.392808 855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:39:30.392872 855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:30.392920 855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:39:30.392954 855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:30.399348 855 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:33437
I20260811 14:39:30.399431 1065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:33437 every 8 connection(s)
I20260811 14:39:30.399881 855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:39:30.400835 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 855
I20260811 14:39:30.400954 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:39:30.401203 2461 external_mini_cluster.cc:1428] Setting key 5634256480320872360050af9357bba7
I20260811 14:39:30.402367 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:33161
--builtin_ntp_servers=127.2.103.84:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 14:39:30.406352 814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:30.400718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:55185 (local address 127.2.103.126:33161)
0811 14:39:30.400825 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:30.400830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:30.401096 (+ 266us) server_negotiation.cc:407] Connection header received
0811 14:39:30.401751 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:30.401752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:30.401764 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:30.401832 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:30.402671 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.403422 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.404621 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.404944 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.405038 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:30.405974 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:30.405975 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:30.405976 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:30.405981 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:30.406001 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:30.406006 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:30.406068 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:30.406206 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 14:39:30.406235 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 14:39:30.406498 1069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:30.400739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:33161 (local address 127.2.103.65:55185)
0811 14:39:30.400939 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:30.400977 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:30.401654 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:30.401887 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:30.401890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:30.401898 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:39:30.402533 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:30.402540 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.403542 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.403543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:30.404529 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:30.404532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.405040 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.405041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:30.405156 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:30.405821 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:39:30.405822 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:39:30.405853 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:39:30.405879 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:39:30.406099 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:39:30.406106 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:39:30.406109 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:39:30.406185 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 14:39:30.406274 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260811 14:39:30.407054 1067 heartbeater.cc:347] Connected to a master server at 127.2.103.126:33161
I20260811 14:39:30.407176 1067 heartbeater.cc:464] Registering TS with master...
I20260811 14:39:30.407371 1067 heartbeater.cc:511] Master 127.2.103.126:33161 requested a full tablet report, sending...
I20260811 14:39:30.407938 748 ts_manager.cc:194] Registered new tserver with Master: 5291b42aec4f4f29a5cc15f25a957f9c (127.2.103.65:33437)
I20260811 14:39:30.408584 748 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.103.65:55185
W20260811 14:39:30.491276 1070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:30.491498 1070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:30.491523 1070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:30.492722 1070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:30.492767 1070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:30.492794 1070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:39:30.494170 1070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:33161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:30.494438 1070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:30.494810 1070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:30.499930 1078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:30.499941 1079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:30.500017 1082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:30.500373 1070 server_base.cc:1034] running on GCE node
I20260811 14:39:30.500547 1070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:30.500752 1070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:30.501969 1070 hybrid_clock.cc:648] HybridClock initialized: now 1786459170501982 us; error 66 us; skew 500 ppm
I20260811 14:39:30.503427 1070 webserver.cc:460] Webserver started at http://127.2.103.66:46827/ using document root <none> and password file <none>
I20260811 14:39:30.503794 1070 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:30.503858 1070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:30.504022 1070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:30.505198 1070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "ae7fccd65c5e42628839a10d1115b237"
format_stamp: "Formatted at 2026-08-11 14:39:30 on dist-test-slave-twwt"
server_key: "16b1d8e44a1dc79c4d08b4fb8ba14391"
server_key_iv: "22e3237198ec9bfcf4f4edf06c5ce66d"
server_key_version: "encryptionkey@0"
I20260811 14:39:30.505584 1070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "ae7fccd65c5e42628839a10d1115b237"
format_stamp: "Formatted at 2026-08-11 14:39:30 on dist-test-slave-twwt"
server_key: "16b1d8e44a1dc79c4d08b4fb8ba14391"
server_key_iv: "22e3237198ec9bfcf4f4edf06c5ce66d"
server_key_version: "encryptionkey@0"
I20260811 14:39:30.507156 1070 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:39:30.507936 1092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:30.508128 1070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:39:30.508226 1070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "ae7fccd65c5e42628839a10d1115b237"
format_stamp: "Formatted at 2026-08-11 14:39:30 on dist-test-slave-twwt"
server_key: "16b1d8e44a1dc79c4d08b4fb8ba14391"
server_key_iv: "22e3237198ec9bfcf4f4edf06c5ce66d"
server_key_version: "encryptionkey@0"
I20260811 14:39:30.508314 1070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:30.524255 1070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:30.524626 1070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:30.524791 1070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:30.525053 1070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:30.525420 1070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:39:30.525477 1070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:30.525520 1070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:39:30.525547 1070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:30.532058 1070 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:43513
I20260811 14:39:30.532142 1277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:43513 every 8 connection(s)
I20260811 14:39:30.532532 1070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:39:30.538090 814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:30.533416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:48589 (local address 127.2.103.126:33161)
0811 14:39:30.533508 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:30.533511 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:30.533743 (+ 232us) server_negotiation.cc:407] Connection header received
0811 14:39:30.534230 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:30.534231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:30.534242 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:30.534293 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:30.535078 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.535579 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.536639 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.536852 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.536907 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:30.537879 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:30.537880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:30.537880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:30.537884 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:30.537896 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:30.537899 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:30.537938 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:30.538006 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 14:39:30.538027 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 14:39:30.538302 1282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:30.533379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:33161 (local address 127.2.103.66:48589)
0811 14:39:30.533605 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:30.533643 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:30.534137 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:30.534359 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:30.534363 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:30.534371 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:39:30.534996 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:30.535002 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.535698 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.535699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:30.536544 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:30.536547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.536936 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.536937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:30.537057 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:30.537708 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:39:30.537708 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:39:30.537749 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:39:30.537780 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:39:30.537940 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:39:30.537943 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:39:30.537945 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:39:30.537993 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 14:39:30.538077 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":108,"threads_started":1}
I20260811 14:39:30.538971 1278 heartbeater.cc:347] Connected to a master server at 127.2.103.126:33161
I20260811 14:39:30.539093 1278 heartbeater.cc:464] Registering TS with master...
I20260811 14:39:30.539294 1278 heartbeater.cc:511] Master 127.2.103.126:33161 requested a full tablet report, sending...
I20260811 14:39:30.539529 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 1070
I20260811 14:39:30.539674 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:39:30.539757 748 ts_manager.cc:194] Registered new tserver with Master: ae7fccd65c5e42628839a10d1115b237 (127.2.103.66:43513)
I20260811 14:39:30.539930 2461 external_mini_cluster.cc:1428] Setting key 3c9bf2ce6037edb667229ed1a18b69bb
I20260811 14:39:30.540225 748 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.103.66:48589
I20260811 14:39:30.541167 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:33161
--builtin_ntp_servers=127.2.103.84:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 14:39:30.630328 1283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:30.630620 1283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:30.630646 1283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:30.631767 1283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:30.631808 1283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:30.631834 1283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:39:30.633183 1283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:33161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:30.633399 1283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:30.633677 1283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:30.639009 1296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:30.639223 1283 server_base.cc:1034] running on GCE node
W20260811 14:39:30.638948 1295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:30.638962 1300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:30.639750 1283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:30.640091 1283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:30.641291 1283 hybrid_clock.cc:648] HybridClock initialized: now 1786459170641284 us; error 42 us; skew 500 ppm
I20260811 14:39:30.642742 1283 webserver.cc:460] Webserver started at http://127.2.103.67:46777/ using document root <none> and password file <none>
I20260811 14:39:30.643042 1283 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:30.643121 1283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:30.643283 1283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:30.644433 1283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "5d436640248e42b983377a3e39b4e94a"
format_stamp: "Formatted at 2026-08-11 14:39:30 on dist-test-slave-twwt"
server_key: "398d01af38682c06afbf121b3a281469"
server_key_iv: "7c7b6d09a5807dd7d48f5a5622772233"
server_key_version: "encryptionkey@0"
I20260811 14:39:30.644827 1283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "5d436640248e42b983377a3e39b4e94a"
format_stamp: "Formatted at 2026-08-11 14:39:30 on dist-test-slave-twwt"
server_key: "398d01af38682c06afbf121b3a281469"
server_key_iv: "7c7b6d09a5807dd7d48f5a5622772233"
server_key_version: "encryptionkey@0"
I20260811 14:39:30.646265 1283 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:39:30.647245 1309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:30.647447 1283 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:39:30.647555 1283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "5d436640248e42b983377a3e39b4e94a"
format_stamp: "Formatted at 2026-08-11 14:39:30 on dist-test-slave-twwt"
server_key: "398d01af38682c06afbf121b3a281469"
server_key_iv: "7c7b6d09a5807dd7d48f5a5622772233"
server_key_version: "encryptionkey@0"
I20260811 14:39:30.647643 1283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:30.672420 1283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:30.672803 1283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:30.672981 1283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:30.673244 1283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:30.673631 1283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:39:30.673707 1283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:30.673760 1283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:39:30.673787 1283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:30.679704 1283 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:35303
I20260811 14:39:30.679786 1492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:35303 every 8 connection(s)
I20260811 14:39:30.680162 1283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:39:30.685581 814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:30.681038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:59253 (local address 127.2.103.126:33161)
0811 14:39:30.681109 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:30.681111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:30.681374 (+ 263us) server_negotiation.cc:407] Connection header received
0811 14:39:30.681884 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:30.681885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:30.681897 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:30.681957 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:30.682750 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.683249 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.684203 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.684397 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.684454 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:30.685339 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:30.685341 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:30.685341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:30.685346 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:30.685359 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:30.685363 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:30.685408 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:30.685496 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 14:39:30.685520 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 14:39:30.685806 1496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:30.681026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:33161 (local address 127.2.103.67:59253)
0811 14:39:30.681238 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:30.681274 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:30.681786 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:30.682044 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:30.682047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:30.682056 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:39:30.682594 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:30.682599 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.683332 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.683333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:30.684117 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:30.684119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.684478 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.684479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:30.684591 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:30.685197 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:39:30.685198 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:39:30.685227 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:39:30.685251 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:39:30.685422 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:39:30.685426 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:39:30.685429 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:39:30.685481 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 14:39:30.685561 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":101,"threads_started":1}
I20260811 14:39:30.686371 1493 heartbeater.cc:347] Connected to a master server at 127.2.103.126:33161
I20260811 14:39:30.686467 1493 heartbeater.cc:464] Registering TS with master...
I20260811 14:39:30.686676 1493 heartbeater.cc:511] Master 127.2.103.126:33161 requested a full tablet report, sending...
I20260811 14:39:30.687089 747 ts_manager.cc:194] Registered new tserver with Master: 5d436640248e42b983377a3e39b4e94a (127.2.103.67:35303)
I20260811 14:39:30.687422 747 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.103.67:59253
I20260811 14:39:30.687814 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 1283
I20260811 14:39:30.688010 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:39:30.688202 2461 external_mini_cluster.cc:1428] Setting key 13a72b851242062c8595383110023e43
I20260811 14:39:30.689116 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:39:30.693382 814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:30.690525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38680 (local address 127.2.103.126:33161)
0811 14:39:30.690594 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:30.690596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:30.690715 (+ 119us) server_negotiation.cc:407] Connection header received
0811 14:39:30.690752 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:30.690753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:30.690762 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:30.690808 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:30.690940 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.691389 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.692131 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.692304 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.692353 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:30.693122 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:30.693123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:30.693123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:30.693127 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:30.693138 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:30.693142 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:30.693207 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:30.693264 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 14:39:30.693273 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 14:39:30.695190 747 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:39:30.696892 747 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:39:30.703737 1506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:30.699359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:43513 (local address 127.0.0.1:37498)
0811 14:39:30.699569 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:30.699611 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:30.699676 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:30.700067 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:30.700069 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:30.700072 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:30.700129 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:30.700130 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.701190 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.701191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:30.702375 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:30.702378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.703417 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.703418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:30.703508 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:30.703524 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 14:39:30.703593 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 14:39:30.703650 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":102,"threads_started":1}
I20260811 14:39:30.703837 1507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:30.699599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:35303 (local address 127.0.0.1:40128)
0811 14:39:30.699848 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:30.699857 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:30.699917 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:30.700222 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:30.700223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:30.700225 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:30.700277 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:30.700280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.701664 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.701665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:30.702728 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:30.702730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.703600 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.703601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:30.703664 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:30.703669 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:39:30.703715 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 14:39:30.703761 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":87,"threads_started":1}
I20260811 14:39:30.704107 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:30.699482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37498 (local address 127.2.103.66:43513)
0811 14:39:30.699839 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:30.699844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:30.699855 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:39:30.699903 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:30.699905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:30.699922 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:30.700005 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:30.700220 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.701090 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.702524 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.703320 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.703404 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:30.703506 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:30.703975 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 14:39:30.704021 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":106,"threads_started":1}
I20260811 14:39:30.704311 1512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:30.699861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40128 (local address 127.2.103.67:35303)
0811 14:39:30.700048 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:30.700052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:30.700062 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:39:30.700101 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:30.700102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:30.700117 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:30.700350 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:30.700408 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.701564 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.702813 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.703504 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.703605 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:30.703700 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:30.703731 (+ 31us) server_negotiation.cc:299] Negotiation successful
0811 14:39:30.703766 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":114,"threads_started":1}
I20260811 14:39:30.704874 1164 tablet_service.cc:1467] Processing CreateTablet for tablet a5a1f5d1c731432f8a308b2948735820 (DEFAULT_TABLE table=test-table [id=33615f6f2a2d4d5ba8c8b3195cb26184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:39:30.705135 1511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:30.699359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:33437 (local address 127.0.0.1:48300)
0811 14:39:30.699734 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:30.699743 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:30.699815 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:30.700364 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:30.700364 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:30.700366 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:30.700415 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:30.700416 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.702657 (+ 2241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.702658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:30.703420 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:30.703421 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.704868 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.704870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:30.704934 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:30.704937 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:39:30.704989 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 14:39:30.705027 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":2,"thread_start_us":91,"threads_started":1}
I20260811 14:39:30.705034 1386 tablet_service.cc:1467] Processing CreateTablet for tablet a5a1f5d1c731432f8a308b2948735820 (DEFAULT_TABLE table=test-table [id=33615f6f2a2d4d5ba8c8b3195cb26184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:39:30.705340 1164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5a1f5d1c731432f8a308b2948735820. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:30.705430 1386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5a1f5d1c731432f8a308b2948735820. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:30.705446 1508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:30.699481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48300 (local address 127.2.103.65:33437)
0811 14:39:30.700150 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:30.700154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:30.700163 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:39:30.700206 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:30.700208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:30.700221 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:30.700292 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:30.700497 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.701582 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.703521 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:30.704677 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:30.704744 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:30.704828 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:30.705304 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 14:39:30.705362 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":109,"threads_started":1}
I20260811 14:39:30.706025 954 tablet_service.cc:1467] Processing CreateTablet for tablet a5a1f5d1c731432f8a308b2948735820 (DEFAULT_TABLE table=test-table [id=33615f6f2a2d4d5ba8c8b3195cb26184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:39:30.706362 954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5a1f5d1c731432f8a308b2948735820. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:30.708580 1513 tablet_bootstrap.cc:492] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a: Bootstrap starting.
I20260811 14:39:30.709312 1513 tablet_bootstrap.cc:654] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:30.709633 1513 log.cc:824] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:30.709699 1516 tablet_bootstrap.cc:492] T a5a1f5d1c731432f8a308b2948735820 P 5291b42aec4f4f29a5cc15f25a957f9c: Bootstrap starting.
I20260811 14:39:30.710002 1515 tablet_bootstrap.cc:492] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237: Bootstrap starting.
I20260811 14:39:30.710415 1513 tablet_bootstrap.cc:492] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a: No bootstrap required, opened a new log
I20260811 14:39:30.710412 1516 tablet_bootstrap.cc:654] T a5a1f5d1c731432f8a308b2948735820 P 5291b42aec4f4f29a5cc15f25a957f9c: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:30.710503 1513 ts_tablet_manager.cc:1397] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:39:30.710697 1515 tablet_bootstrap.cc:654] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:30.710781 1516 log.cc:824] T a5a1f5d1c731432f8a308b2948735820 P 5291b42aec4f4f29a5cc15f25a957f9c: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:30.711018 1515 log.cc:824] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:30.711627 1516 tablet_bootstrap.cc:492] T a5a1f5d1c731432f8a308b2948735820 P 5291b42aec4f4f29a5cc15f25a957f9c: No bootstrap required, opened a new log
I20260811 14:39:30.711714 1516 ts_tablet_manager.cc:1397] T a5a1f5d1c731432f8a308b2948735820 P 5291b42aec4f4f29a5cc15f25a957f9c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 14:39:30.711870 1515 tablet_bootstrap.cc:492] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237: No bootstrap required, opened a new log
I20260811 14:39:30.711971 1515 ts_tablet_manager.cc:1397] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 14:39:30.713626 1516 raft_consensus.cc:348] T a5a1f5d1c731432f8a308b2948735820 P 5291b42aec4f4f29a5cc15f25a957f9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5291b42aec4f4f29a5cc15f25a957f9c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 33437 } } peers { permanent_uuid: "5d436640248e42b983377a3e39b4e94a" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35303 } } peers { permanent_uuid: "ae7fccd65c5e42628839a10d1115b237" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 43513 } }
I20260811 14:39:30.713789 1516 raft_consensus.cc:374] T a5a1f5d1c731432f8a308b2948735820 P 5291b42aec4f4f29a5cc15f25a957f9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:30.713836 1516 raft_consensus.cc:729] T a5a1f5d1c731432f8a308b2948735820 P 5291b42aec4f4f29a5cc15f25a957f9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5291b42aec4f4f29a5cc15f25a957f9c, State: Initialized, Role: FOLLOWER
I20260811 14:39:30.713953 1516 consensus_queue.cc:260] T a5a1f5d1c731432f8a308b2948735820 P 5291b42aec4f4f29a5cc15f25a957f9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5291b42aec4f4f29a5cc15f25a957f9c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 33437 } } peers { permanent_uuid: "5d436640248e42b983377a3e39b4e94a" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35303 } } peers { permanent_uuid: "ae7fccd65c5e42628839a10d1115b237" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 43513 } }
I20260811 14:39:30.713932 1513 raft_consensus.cc:348] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5291b42aec4f4f29a5cc15f25a957f9c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 33437 } } peers { permanent_uuid: "5d436640248e42b983377a3e39b4e94a" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35303 } } peers { permanent_uuid: "ae7fccd65c5e42628839a10d1115b237" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 43513 } }
I20260811 14:39:30.714104 1513 raft_consensus.cc:374] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:30.714138 1513 raft_consensus.cc:729] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d436640248e42b983377a3e39b4e94a, State: Initialized, Role: FOLLOWER
I20260811 14:39:30.714262 1513 consensus_queue.cc:260] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5291b42aec4f4f29a5cc15f25a957f9c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 33437 } } peers { permanent_uuid: "5d436640248e42b983377a3e39b4e94a" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35303 } } peers { permanent_uuid: "ae7fccd65c5e42628839a10d1115b237" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 43513 } }
I20260811 14:39:30.714286 1516 ts_tablet_manager.cc:1428] T a5a1f5d1c731432f8a308b2948735820 P 5291b42aec4f4f29a5cc15f25a957f9c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 14:39:30.714509 1067 heartbeater.cc:503] Master 127.2.103.126:33161 was elected leader, sending a full tablet report...
I20260811 14:39:30.714686 1493 heartbeater.cc:503] Master 127.2.103.126:33161 was elected leader, sending a full tablet report...
I20260811 14:39:30.714715 1513 ts_tablet_manager.cc:1428] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260811 14:39:30.714686 1515 raft_consensus.cc:348] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5291b42aec4f4f29a5cc15f25a957f9c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 33437 } } peers { permanent_uuid: "5d436640248e42b983377a3e39b4e94a" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35303 } } peers { permanent_uuid: "ae7fccd65c5e42628839a10d1115b237" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 43513 } }
I20260811 14:39:30.714847 1515 raft_consensus.cc:374] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:30.714898 1515 raft_consensus.cc:729] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae7fccd65c5e42628839a10d1115b237, State: Initialized, Role: FOLLOWER
I20260811 14:39:30.715021 1515 consensus_queue.cc:260] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5291b42aec4f4f29a5cc15f25a957f9c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 33437 } } peers { permanent_uuid: "5d436640248e42b983377a3e39b4e94a" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35303 } } peers { permanent_uuid: "ae7fccd65c5e42628839a10d1115b237" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 43513 } }
I20260811 14:39:30.715354 1515 ts_tablet_manager.cc:1428] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260811 14:39:30.715538 1278 heartbeater.cc:503] Master 127.2.103.126:33161 was elected leader, sending a full tablet report...
W20260811 14:39:30.783200 1281 tablet.cc:2369] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:39:30.900717 1068 tablet.cc:2369] T a5a1f5d1c731432f8a308b2948735820 P 5291b42aec4f4f29a5cc15f25a957f9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:39:30.930823 1494 tablet.cc:2369] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:39:31.085217 1528 raft_consensus.cc:482] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:39:31.085359 1528 raft_consensus.cc:504] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5291b42aec4f4f29a5cc15f25a957f9c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 33437 } } peers { permanent_uuid: "5d436640248e42b983377a3e39b4e94a" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35303 } } peers { permanent_uuid: "ae7fccd65c5e42628839a10d1115b237" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 43513 } }
I20260811 14:39:31.085687 1528 leader_election.cc:288] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5291b42aec4f4f29a5cc15f25a957f9c (127.2.103.65:33437), 5d436640248e42b983377a3e39b4e94a (127.2.103.67:35303)
I20260811 14:39:31.090014 1508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:31.086048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:52185 (local address 127.2.103.65:33437)
0811 14:39:31.086130 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:31.086133 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:31.086177 (+ 44us) server_negotiation.cc:407] Connection header received
0811 14:39:31.086345 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:31.086347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:31.086359 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:31.086424 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:31.086688 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:31.087458 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:31.089038 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:31.089622 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:31.089699 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:31.089708 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:31.089894 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 14:39:31.089940 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 14:39:31.090019 1512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:31.086212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:35183 (local address 127.2.103.67:35303)
0811 14:39:31.086267 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:31.086269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:31.086279 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:39:31.086368 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:31.086369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:31.086378 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:31.086440 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:31.086688 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:31.087479 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:31.089070 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:31.089622 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:31.089698 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:31.089708 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:31.089918 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 14:39:31.089962 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 14:39:31.090035 1529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:31.085939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:33437 (local address 127.2.103.66:52185)
0811 14:39:31.086093 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:31.086108 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:31.086201 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:31.086467 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:31.086468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:31.086471 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:31.086589 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:31.086593 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:31.087558 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:31.087559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:31.088942 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:31.088946 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:31.089712 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:31.089712 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:31.089797 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:31.089806 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 14:39:31.089867 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 14:39:31.089921 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":100,"thread_start_us":68,"threads_started":1}
I20260811 14:39:31.090046 1530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:31.086125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:35303 (local address 127.2.103.66:35183)
0811 14:39:31.086215 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:31.086225 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:31.086280 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:31.086467 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:31.086468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:31.086470 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:31.086589 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:31.086593 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:31.087558 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:31.087559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:31.088981 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:31.088983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:31.089710 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:31.089711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:31.089797 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:31.089805 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 14:39:31.089901 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 14:39:31.089928 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":78,"thread_start_us":39,"threads_started":1}
I20260811 14:39:31.090440 990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5a1f5d1c731432f8a308b2948735820" candidate_uuid: "ae7fccd65c5e42628839a10d1115b237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5291b42aec4f4f29a5cc15f25a957f9c" is_pre_election: true
I20260811 14:39:31.090659 990 raft_consensus.cc:2454] T a5a1f5d1c731432f8a308b2948735820 P 5291b42aec4f4f29a5cc15f25a957f9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae7fccd65c5e42628839a10d1115b237 in term 0.
I20260811 14:39:31.090719 1419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5a1f5d1c731432f8a308b2948735820" candidate_uuid: "ae7fccd65c5e42628839a10d1115b237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d436640248e42b983377a3e39b4e94a" is_pre_election: true
I20260811 14:39:31.090878 1419 raft_consensus.cc:2454] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae7fccd65c5e42628839a10d1115b237 in term 0.
I20260811 14:39:31.090919 1095 leader_election.cc:302] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5291b42aec4f4f29a5cc15f25a957f9c, ae7fccd65c5e42628839a10d1115b237; no voters:
I20260811 14:39:31.091089 1528 raft_consensus.cc:2781] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:39:31.091163 1528 raft_consensus.cc:482] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:39:31.091200 1528 raft_consensus.cc:3037] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:31.091982 1528 raft_consensus.cc:504] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5291b42aec4f4f29a5cc15f25a957f9c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 33437 } } peers { permanent_uuid: "5d436640248e42b983377a3e39b4e94a" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35303 } } peers { permanent_uuid: "ae7fccd65c5e42628839a10d1115b237" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 43513 } }
I20260811 14:39:31.092137 1528 leader_election.cc:288] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [CANDIDATE]: Term 1 election: Requested vote from peers 5291b42aec4f4f29a5cc15f25a957f9c (127.2.103.65:33437), 5d436640248e42b983377a3e39b4e94a (127.2.103.67:35303)
I20260811 14:39:31.092299 990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5a1f5d1c731432f8a308b2948735820" candidate_uuid: "ae7fccd65c5e42628839a10d1115b237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5291b42aec4f4f29a5cc15f25a957f9c"
I20260811 14:39:31.092393 990 raft_consensus.cc:3037] T a5a1f5d1c731432f8a308b2948735820 P 5291b42aec4f4f29a5cc15f25a957f9c [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:31.092355 1419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5a1f5d1c731432f8a308b2948735820" candidate_uuid: "ae7fccd65c5e42628839a10d1115b237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d436640248e42b983377a3e39b4e94a"
I20260811 14:39:31.092463 1419 raft_consensus.cc:3037] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:31.093199 990 raft_consensus.cc:2454] T a5a1f5d1c731432f8a308b2948735820 P 5291b42aec4f4f29a5cc15f25a957f9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae7fccd65c5e42628839a10d1115b237 in term 1.
I20260811 14:39:31.093338 1419 raft_consensus.cc:2454] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae7fccd65c5e42628839a10d1115b237 in term 1.
I20260811 14:39:31.093514 1095 leader_election.cc:302] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d436640248e42b983377a3e39b4e94a, ae7fccd65c5e42628839a10d1115b237; no voters:
I20260811 14:39:31.093602 1528 raft_consensus.cc:2781] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:31.093778 1528 raft_consensus.cc:686] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [term 1 LEADER]: Becoming Leader. State: Replica: ae7fccd65c5e42628839a10d1115b237, State: Running, Role: LEADER
I20260811 14:39:31.093892 1528 consensus_queue.cc:237] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5291b42aec4f4f29a5cc15f25a957f9c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 33437 } } peers { permanent_uuid: "5d436640248e42b983377a3e39b4e94a" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35303 } } peers { permanent_uuid: "ae7fccd65c5e42628839a10d1115b237" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 43513 } }
I20260811 14:39:31.094798 746 catalog_manager.cc:5495] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae7fccd65c5e42628839a10d1115b237 (127.2.103.66). New cstate: current_term: 1 leader_uuid: "ae7fccd65c5e42628839a10d1115b237" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5291b42aec4f4f29a5cc15f25a957f9c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 33437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d436640248e42b983377a3e39b4e94a" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae7fccd65c5e42628839a10d1115b237" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 43513 } health_report { overall_health: HEALTHY } } }
I20260811 14:39:31.145452 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:31.142969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37504 (local address 127.2.103.66:43513)
0811 14:39:31.143039 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:31.143042 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:31.143051 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:39:31.143102 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:31.143103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:31.143126 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:31.143211 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:39:31.143372 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:31.143858 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:31.144635 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:31.144845 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:31.144926 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:31.145088 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:39:31.145186 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:39:31.145297 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:31.145359 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 14:39:31.145371 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 14:39:31.147468 990 raft_consensus.cc:1261] T a5a1f5d1c731432f8a308b2948735820 P 5291b42aec4f4f29a5cc15f25a957f9c [term 1 FOLLOWER]: Refusing update from remote peer ae7fccd65c5e42628839a10d1115b237: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:39:31.147496 1419 raft_consensus.cc:1261] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a [term 1 FOLLOWER]: Refusing update from remote peer ae7fccd65c5e42628839a10d1115b237: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:39:31.147912 1533 consensus_queue.cc:1035] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5291b42aec4f4f29a5cc15f25a957f9c" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 33437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:39:31.148057 1528 consensus_queue.cc:1035] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d436640248e42b983377a3e39b4e94a" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 35303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:39:31.151537 1541 mvcc.cc:205] Tried to move back new op lower bound from 7317336765016899584 to 7317336764801318912. Current Snapshot: MvccSnapshot[applied={T|T < 7317336765016899584 or (T in {7317336765016899584})}]
I20260811 14:39:31.153468 747 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:39:31.153573 747 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:39:31.154428 747 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0fc7e469e8e844309a6a50dfab256682: Sending DeleteTablet for 3 replicas of tablet a5a1f5d1c731432f8a308b2948735820
I20260811 14:39:31.154790 1164 tablet_service.cc:1514] Processing DeleteTablet for tablet a5a1f5d1c731432f8a308b2948735820 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:39:31 UTC) from {username='slave'} at 127.0.0.1:37498
I20260811 14:39:31.154796 1386 tablet_service.cc:1514] Processing DeleteTablet for tablet a5a1f5d1c731432f8a308b2948735820 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:39:31 UTC) from {username='slave'} at 127.0.0.1:40128
I20260811 14:39:31.154862 954 tablet_service.cc:1514] Processing DeleteTablet for tablet a5a1f5d1c731432f8a308b2948735820 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:39:31 UTC) from {username='slave'} at 127.0.0.1:48300
I20260811 14:39:31.155072 1559 tablet_replica.cc:330] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237: stopping tablet replica
I20260811 14:39:31.155189 1559 raft_consensus.cc:2229] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:39:31.155257 1561 tablet_replica.cc:330] T a5a1f5d1c731432f8a308b2948735820 P 5291b42aec4f4f29a5cc15f25a957f9c: stopping tablet replica
I20260811 14:39:31.155331 1559 raft_consensus.cc:2258] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:39:31.155350 1561 raft_consensus.cc:2229] T a5a1f5d1c731432f8a308b2948735820 P 5291b42aec4f4f29a5cc15f25a957f9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:39:31.155447 1561 raft_consensus.cc:2258] T a5a1f5d1c731432f8a308b2948735820 P 5291b42aec4f4f29a5cc15f25a957f9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:39:31.155452 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 855
I20260811 14:39:31.158881 1559 ts_tablet_manager.cc:1905] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:39:31.160670 1559 ts_tablet_manager.cc:1918] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:39:31.160737 1559 log.cc:1196] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/a5a1f5d1c731432f8a308b2948735820
I20260811 14:39:31.161002 1559 ts_tablet_manager.cc:1939] T a5a1f5d1c731432f8a308b2948735820 P ae7fccd65c5e42628839a10d1115b237: Deleting consensus metadata
I20260811 14:39:31.161000 1560 tablet_replica.cc:330] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a: stopping tablet replica
I20260811 14:39:31.161128 1560 raft_consensus.cc:2229] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:39:31.161252 1560 raft_consensus.cc:2258] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:39:31.166261 1560 ts_tablet_manager.cc:1905] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:39:31.168579 1560 ts_tablet_manager.cc:1918] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:39:31.168653 1560 log.cc:1196] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/a5a1f5d1c731432f8a308b2948735820
I20260811 14:39:31.168845 1560 ts_tablet_manager.cc:1939] T a5a1f5d1c731432f8a308b2948735820 P 5d436640248e42b983377a3e39b4e94a: Deleting consensus metadata
I20260811 14:39:31.169329 728 catalog_manager.cc:4841] TS 5d436640248e42b983377a3e39b4e94a (127.2.103.67:35303): tablet a5a1f5d1c731432f8a308b2948735820 (table test-table [id=33615f6f2a2d4d5ba8c8b3195cb26184]) successfully deleted
I20260811 14:39:31.175451 729 catalog_manager.cc:4841] TS ae7fccd65c5e42628839a10d1115b237 (127.2.103.66:43513): tablet a5a1f5d1c731432f8a308b2948735820 (table test-table [id=33615f6f2a2d4d5ba8c8b3195cb26184]) successfully deleted
W20260811 14:39:31.179647 728 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.103.65:33437 (error 108)
I20260811 14:39:31.180254 1511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:31.180096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:33437 (local address 127.0.0.1:48306)
0811 14:39:31.180165 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:31.180214 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.65:33437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 14:39:31.180334 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 1070
W20260811 14:39:31.185681 728 catalog_manager.cc:4584] TS 5291b42aec4f4f29a5cc15f25a957f9c (127.2.103.65:33437): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a5a1f5d1c731432f8a308b2948735820: Network error: Client connection negotiation failed: client connection to 127.2.103.65:33437: connect: Connection refused (error 111)
I20260811 14:39:31.190429 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 1283
I20260811 14:39:31.206827 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 697
2026-08-11T14:39:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1145 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:39:31 dist-test-slave-twwt krb5kdc[1568](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:39:31 dist-test-slave-twwt krb5kdc[1568](info): set up 2 sockets
Aug 11 14:39:31 dist-test-slave-twwt krb5kdc[1568](info): commencing operation
krb5kdc: starting...
W20260811 14:39:32.511361 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:39:32 dist-test-slave-twwt krb5kdc[1568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:39:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:39:32.647681 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:39681
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:39681
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:32.735584 1583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:32.735801 1583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:32.735826 1583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:32.736986 1583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:32.737030 1583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:32.737046 1583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:32.737058 1583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:32.737071 1583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:32.738584 1583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:39681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:39681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:32.738883 1583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:32.739125 1583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:32.744190 1592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:32.744202 1595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:32.744365 1583 server_base.cc:1034] running on GCE node
W20260811 14:39:32.744385 1589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:32.744740 1583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:32.745087 1583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:32.746315 1583 hybrid_clock.cc:648] HybridClock initialized: now 1786459172746270 us; error 44 us; skew 500 ppm
Aug 11 14:39:32 dist-test-slave-twwt krb5kdc[1568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459172, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:32.749019 1583 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:39:32.749475 1583 webserver.cc:460] Webserver started at http://127.2.103.126:43573/ using document root <none> and password file <none>
I20260811 14:39:32.749706 1583 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:32.749773 1583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:32.749927 1583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:32.751111 1583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "07b8dcec27fd465cb8e795b6afb324d7"
format_stamp: "Formatted at 2026-08-11 14:39:32 on dist-test-slave-twwt"
server_key: "653e5072b96b67d6ede434b725dd3b14"
server_key_iv: "808a26c831e8efeb0fe3c72d56d44083"
server_key_version: "encryptionkey@0"
I20260811 14:39:32.751518 1583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "07b8dcec27fd465cb8e795b6afb324d7"
format_stamp: "Formatted at 2026-08-11 14:39:32 on dist-test-slave-twwt"
server_key: "653e5072b96b67d6ede434b725dd3b14"
server_key_iv: "808a26c831e8efeb0fe3c72d56d44083"
server_key_version: "encryptionkey@0"
I20260811 14:39:32.752969 1583 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:39:32.753805 1607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:32.754000 1583 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:32.754113 1583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "07b8dcec27fd465cb8e795b6afb324d7"
format_stamp: "Formatted at 2026-08-11 14:39:32 on dist-test-slave-twwt"
server_key: "653e5072b96b67d6ede434b725dd3b14"
server_key_iv: "808a26c831e8efeb0fe3c72d56d44083"
server_key_version: "encryptionkey@0"
I20260811 14:39:32.754192 1583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:32.772734 1583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:32.775764 1583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:32.775965 1583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:32.781376 1583 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:39681
I20260811 14:39:32.781420 1699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:39681 every 8 connection(s)
I20260811 14:39:32.781822 1583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:32.783463 1700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:32.784790 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 1583
I20260811 14:39:32.785029 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:32.785617 2461 external_mini_cluster.cc:1428] Setting key 4f147a5893414dfcc7ce1e9d0ff7113e
I20260811 14:39:32.790195 1700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7: Bootstrap starting.
I20260811 14:39:32.791072 1700 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:32.791468 1700 log.cc:824] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:32.792385 1700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7: No bootstrap required, opened a new log
I20260811 14:39:32.795038 1700 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b8dcec27fd465cb8e795b6afb324d7" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 39681 } }
I20260811 14:39:32.795195 1700 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:32.795220 1700 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b8dcec27fd465cb8e795b6afb324d7, State: Initialized, Role: FOLLOWER
I20260811 14:39:32.795374 1700 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b8dcec27fd465cb8e795b6afb324d7" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 39681 } }
I20260811 14:39:32.795473 1700 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:32.795521 1700 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:32.795578 1700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:32.796268 1700 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b8dcec27fd465cb8e795b6afb324d7" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 39681 } }
Aug 11 14:39:32 dist-test-slave-twwt krb5kdc[1568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:39:32.796429 1700 leader_election.cc:302] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07b8dcec27fd465cb8e795b6afb324d7; no voters:
I20260811 14:39:32.796650 1700 leader_election.cc:288] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:32.796689 1706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:32.796938 1706 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7 [term 1 LEADER]: Becoming Leader. State: Replica: 07b8dcec27fd465cb8e795b6afb324d7, State: Running, Role: LEADER
I20260811 14:39:32.797049 1706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b8dcec27fd465cb8e795b6afb324d7" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 39681 } }
I20260811 14:39:32.797066 1700 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:32.797516 1709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07b8dcec27fd465cb8e795b6afb324d7. Latest consensus state: current_term: 1 leader_uuid: "07b8dcec27fd465cb8e795b6afb324d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b8dcec27fd465cb8e795b6afb324d7" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 39681 } } }
I20260811 14:39:32.797629 1709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:32.797516 1708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07b8dcec27fd465cb8e795b6afb324d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b8dcec27fd465cb8e795b6afb324d7" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 39681 } } }
I20260811 14:39:32.797863 1720 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:32.797916 1708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:32.798427 1720 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:32.801231 1720 catalog_manager.cc:1347] Generated new cluster ID: 786a73f4802f44f0a6f55626931aa26d
I20260811 14:39:32.801291 1720 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:39:32.802194 1703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:32.787108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52456 (local address 127.2.103.126:39681)
0811 14:39:32.787460 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:32.787470 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:32.787516 (+ 46us) server_negotiation.cc:407] Connection header received
0811 14:39:32.789468 (+ 1952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:32.789581 (+ 113us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:32.789954 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:32.790152 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:32.790977 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:32.792526 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:32.793388 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:32.794094 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:32.794226 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:32.797216 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:32.797220 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:32.797221 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:32.797236 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:32.800203 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:32.800712 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:32.800715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:32.800715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:32.800888 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:32.801148 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:32.801149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:32.801149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:32.801507 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:32.801666 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:32.801816 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 14:39:32.801966 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":165,"threads_started":1}
I20260811 14:39:32.814098 1720 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:32.814836 1720 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:32.821213 1720 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7: Generated new TSK 0
I20260811 14:39:32.821447 1720 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.103.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
Entry for principal kudu/127.2.103.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab.
I20260811 14:39:32.859864 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:39681
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:32.955348 1742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:32.955582 1742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:32.955608 1742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:32.956763 1742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:32.956807 1742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:32.956835 1742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:39:32.958482 1742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.65.keytab
--principal=kudu/127.2.103.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:39681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:32.958776 1742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:32.958984 1742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:32.964197 1753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:32.964233 1751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:32.964447 1742 server_base.cc:1034] running on GCE node
W20260811 14:39:32.964232 1755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:32.964700 1742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:32.964920 1742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:32.966063 1742 hybrid_clock.cc:648] HybridClock initialized: now 1786459172966051 us; error 29 us; skew 500 ppm
Aug 11 14:39:32 dist-test-slave-twwt krb5kdc[1568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459172, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:32.968756 1742 init.cc:377] Logged in from keytab as kudu/127.2.103.65@KRBTEST.COM (short username kudu)
I20260811 14:39:32.969260 1742 webserver.cc:460] Webserver started at http://127.2.103.65:35495/ using document root <none> and password file <none>
I20260811 14:39:32.969527 1742 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:32.969599 1742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:32.969750 1742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:32.970937 1742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "6beb6ff029cf44fc8f6341bc3be499ba"
format_stamp: "Formatted at 2026-08-11 14:39:32 on dist-test-slave-twwt"
server_key: "d6d5f0f1eac34163f978087b3cc0d422"
server_key_iv: "bad4865f1b81f2dbc1169e4f882781ae"
server_key_version: "encryptionkey@0"
I20260811 14:39:32.971320 1742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "6beb6ff029cf44fc8f6341bc3be499ba"
format_stamp: "Formatted at 2026-08-11 14:39:32 on dist-test-slave-twwt"
server_key: "d6d5f0f1eac34163f978087b3cc0d422"
server_key_iv: "bad4865f1b81f2dbc1169e4f882781ae"
server_key_version: "encryptionkey@0"
I20260811 14:39:32.972820 1742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:39:32.973632 1767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:32.973820 1742 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:32.973937 1742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "6beb6ff029cf44fc8f6341bc3be499ba"
format_stamp: "Formatted at 2026-08-11 14:39:32 on dist-test-slave-twwt"
server_key: "d6d5f0f1eac34163f978087b3cc0d422"
server_key_iv: "bad4865f1b81f2dbc1169e4f882781ae"
server_key_version: "encryptionkey@0"
I20260811 14:39:32.974023 1742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:32.998427 1742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:33.000949 1742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:33.001159 1742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:33.001452 1742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:33.001812 1742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:39:33.001865 1742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:33.001904 1742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:39:33.001931 1742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:33.008203 1742 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:46099
I20260811 14:39:33.008261 1973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:46099 every 8 connection(s)
I20260811 14:39:33.008625 1742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:39:33 dist-test-slave-twwt krb5kdc[1568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459172, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.65@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:39:33.017319 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 1742
I20260811 14:39:33.017474 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:39:33.017721 2461 external_mini_cluster.cc:1428] Setting key fcffdadbc0e96b49d352225116eafe08
I20260811 14:39:33.019343 1703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.009437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:43783 (local address 127.2.103.126:39681)
0811 14:39:33.009498 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:33.009500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:33.009705 (+ 205us) server_negotiation.cc:407] Connection header received
0811 14:39:33.010188 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:33.010189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:33.010199 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:33.010261 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:33.011290 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.011833 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.012957 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.013160 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.013228 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.015702 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:33.015703 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:33.015704 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:33.015712 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:33.017471 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:33.018028 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:33.018030 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:33.018031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:33.018107 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:33.018553 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:33.018554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:33.018555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:33.018742 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:33.018829 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:33.019139 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 14:39:33.019243 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 14:39:33.019569 1981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.009406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:39681 (local address 127.2.103.65:43783)
0811 14:39:33.009568 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:33.009612 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:33.010105 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:33.010317 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:33.010321 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:33.010947 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:39:33.011161 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:33.011167 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.011950 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.011951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:33.012855 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:33.012858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.013249 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.013250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:33.013385 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.014001 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:39:33.014010 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:39:33.015588 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:39:33.017583 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:33.017593 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:33.017596 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:33.017929 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:33.018203 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:33.018204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:33.018204 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:33.018374 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:33.018835 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:39:33.018844 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:39:33.018991 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 14:39:33.019103 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 14:39:33.019296 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260811 14:39:33.020128 1976 heartbeater.cc:347] Connected to a master server at 127.2.103.126:39681
I20260811 14:39:33.020262 1976 heartbeater.cc:464] Registering TS with master...
I20260811 14:39:33.020490 1976 heartbeater.cc:511] Master 127.2.103.126:39681 requested a full tablet report, sending...
I20260811 14:39:33.021255 1635 ts_manager.cc:194] Registered new tserver with Master: 6beb6ff029cf44fc8f6341bc3be499ba (127.2.103.65:46099)
I20260811 14:39:33.022248 1635 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.65@KRBTEST.COM'} at 127.2.103.65:43783
WARNING: no policy specified for kudu/127.2.103.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
Entry for principal kudu/127.2.103.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab.
I20260811 14:39:33.049643 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:39681
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:33.136366 1985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:33.136592 1985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:33.136620 1985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:33.137863 1985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:33.137907 1985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:33.137936 1985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:39:33.139565 1985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.66.keytab
--principal=kudu/127.2.103.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:39681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:33.139837 1985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:33.140103 1985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:33.145335 1991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:33.145352 1990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:33.145352 1994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:33.145624 1985 server_base.cc:1034] running on GCE node
I20260811 14:39:33.145807 1985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:33.146025 1985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:33.147177 1985 hybrid_clock.cc:648] HybridClock initialized: now 1786459173147164 us; error 29 us; skew 500 ppm
Aug 11 14:39:33 dist-test-slave-twwt krb5kdc[1568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459173, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:33.149936 1985 init.cc:377] Logged in from keytab as kudu/127.2.103.66@KRBTEST.COM (short username kudu)
I20260811 14:39:33.150369 1985 webserver.cc:460] Webserver started at http://127.2.103.66:43787/ using document root <none> and password file <none>
I20260811 14:39:33.150590 1985 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:33.150697 1985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:33.150803 1985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:33.151865 1985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "1f38dd66c97e4c7da71852c360115d2c"
format_stamp: "Formatted at 2026-08-11 14:39:33 on dist-test-slave-twwt"
server_key: "5c712ff49ba50a1e8f53acd78ec9324a"
server_key_iv: "1f3f263e0aba20c46529728a8e8c7a4f"
server_key_version: "encryptionkey@0"
I20260811 14:39:33.152213 1985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "1f38dd66c97e4c7da71852c360115d2c"
format_stamp: "Formatted at 2026-08-11 14:39:33 on dist-test-slave-twwt"
server_key: "5c712ff49ba50a1e8f53acd78ec9324a"
server_key_iv: "1f3f263e0aba20c46529728a8e8c7a4f"
server_key_version: "encryptionkey@0"
I20260811 14:39:33.153609 1985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:39:33.154498 2003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:33.154742 1985 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:33.154889 1985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "1f38dd66c97e4c7da71852c360115d2c"
format_stamp: "Formatted at 2026-08-11 14:39:33 on dist-test-slave-twwt"
server_key: "5c712ff49ba50a1e8f53acd78ec9324a"
server_key_iv: "1f3f263e0aba20c46529728a8e8c7a4f"
server_key_version: "encryptionkey@0"
I20260811 14:39:33.154978 1985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:33.166067 1985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:33.168417 1985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:33.168567 1985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:33.168877 1985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:33.169252 1985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:39:33.169288 1985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:33.169335 1985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:39:33.169365 1985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:33.176779 1985 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:42929
I20260811 14:39:33.176828 2212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:42929 every 8 connection(s)
I20260811 14:39:33.177255 1985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
Aug 11 14:39:33 dist-test-slave-twwt krb5kdc[1568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459173, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.66@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:39:33.186141 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 1985
I20260811 14:39:33.186286 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:39:33.186534 2461 external_mini_cluster.cc:1428] Setting key 765b05deb18f2034a57986fda4e31860
I20260811 14:39:33.187686 1703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.178055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:57613 (local address 127.2.103.126:39681)
0811 14:39:33.178117 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:33.178119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:33.178414 (+ 295us) server_negotiation.cc:407] Connection header received
0811 14:39:33.178992 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:33.178993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:33.179005 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:33.179063 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:33.180073 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.180583 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.181780 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.182009 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.182071 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.184468 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:33.184470 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:33.184470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:33.184478 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:33.185722 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:33.186365 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:33.186366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:33.186367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:33.186439 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:33.186934 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:33.186935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:33.186935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:33.187055 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:33.187141 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:33.187501 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 14:39:33.187596 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 14:39:33.187939 2220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.178091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:39681 (local address 127.2.103.66:57613)
0811 14:39:33.178275 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:33.178320 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:33.178895 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:33.179140 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:33.179143 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:33.179781 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:39:33.179991 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:33.179996 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.180732 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.180733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:33.181669 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:33.181672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.182113 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.182114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:33.182231 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.182853 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:39:33.182862 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:39:33.184369 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:39:33.185971 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:33.185978 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:33.185981 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:33.186234 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:33.186537 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:33.186538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:33.186539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:33.186743 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:33.187144 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:39:33.187153 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:39:33.187336 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 14:39:33.187462 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 14:39:33.187643 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":100,"threads_started":1}
I20260811 14:39:33.188544 2214 heartbeater.cc:347] Connected to a master server at 127.2.103.126:39681
I20260811 14:39:33.188658 2214 heartbeater.cc:464] Registering TS with master...
I20260811 14:39:33.188862 2214 heartbeater.cc:511] Master 127.2.103.126:39681 requested a full tablet report, sending...
I20260811 14:39:33.189317 1635 ts_manager.cc:194] Registered new tserver with Master: 1f38dd66c97e4c7da71852c360115d2c (127.2.103.66:42929)
I20260811 14:39:33.189713 1635 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.66@KRBTEST.COM'} at 127.2.103.66:57613
WARNING: no policy specified for kudu/127.2.103.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
Entry for principal kudu/127.2.103.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab.
I20260811 14:39:33.218173 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:39681
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:44623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:33.306103 2226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:33.306380 2226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:33.306420 2226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:33.307710 2226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:33.307790 2226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:33.307842 2226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:39:33.309489 2226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:44623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.67.keytab
--principal=kudu/127.2.103.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:39681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:33.309796 2226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:33.310063 2226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:33.315349 2232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:33.315416 2233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:33.315390 2235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:33.315631 2226 server_base.cc:1034] running on GCE node
I20260811 14:39:33.315862 2226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:33.316149 2226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:33.317323 2226 hybrid_clock.cc:648] HybridClock initialized: now 1786459173317323 us; error 42 us; skew 500 ppm
Aug 11 14:39:33 dist-test-slave-twwt krb5kdc[1568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459173, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:33.320108 2226 init.cc:377] Logged in from keytab as kudu/127.2.103.67@KRBTEST.COM (short username kudu)
I20260811 14:39:33.320605 2226 webserver.cc:460] Webserver started at http://127.2.103.67:34007/ using document root <none> and password file <none>
I20260811 14:39:33.320927 2226 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:33.321004 2226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:33.321154 2226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:33.322228 2226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "45c5d0d679c34efda229cc99adef9f35"
format_stamp: "Formatted at 2026-08-11 14:39:33 on dist-test-slave-twwt"
server_key: "dc2f921efca04ba5b54c141e94131c6a"
server_key_iv: "f9fcd88efd5a1623ed1be5e254d0e2b8"
server_key_version: "encryptionkey@0"
I20260811 14:39:33.322662 2226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "45c5d0d679c34efda229cc99adef9f35"
format_stamp: "Formatted at 2026-08-11 14:39:33 on dist-test-slave-twwt"
server_key: "dc2f921efca04ba5b54c141e94131c6a"
server_key_iv: "f9fcd88efd5a1623ed1be5e254d0e2b8"
server_key_version: "encryptionkey@0"
I20260811 14:39:33.324254 2226 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:39:33.325073 2250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:33.325270 2226 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:39:33.325378 2226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "45c5d0d679c34efda229cc99adef9f35"
format_stamp: "Formatted at 2026-08-11 14:39:33 on dist-test-slave-twwt"
server_key: "dc2f921efca04ba5b54c141e94131c6a"
server_key_iv: "f9fcd88efd5a1623ed1be5e254d0e2b8"
server_key_version: "encryptionkey@0"
I20260811 14:39:33.325469 2226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:33.337052 2226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:33.339154 2226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:33.339330 2226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:33.339586 2226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:33.339936 2226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:39:33.339993 2226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:33.340040 2226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:39:33.340086 2226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:33.346542 2226 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:42395
I20260811 14:39:33.346596 2436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:42395 every 8 connection(s)
I20260811 14:39:33.346984 2226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
Aug 11 14:39:33 dist-test-slave-twwt krb5kdc[1568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459173, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.67@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:39:33.354635 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 2226
I20260811 14:39:33.354782 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:39:33.355000 2461 external_mini_cluster.cc:1428] Setting key f605b834d68a618f9f663e34be393640
I20260811 14:39:33.356993 1703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.347898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:53661 (local address 127.2.103.126:39681)
0811 14:39:33.347947 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:33.347949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:33.348113 (+ 164us) server_negotiation.cc:407] Connection header received
0811 14:39:33.348614 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:33.348616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:33.348628 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:33.348698 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:33.349710 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.350254 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.351308 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.351506 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.351567 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.353856 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:33.353858 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:33.353858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:33.353865 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:33.355559 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:33.356108 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:33.356109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:33.356109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:33.356164 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:33.356403 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:33.356404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:33.356404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:33.356513 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:33.356561 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:33.356782 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 14:39:33.356894 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:39:33.357334 2441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.347774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:39681 (local address 127.2.103.67:53661)
0811 14:39:33.347951 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:33.347994 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:33.348498 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:33.348780 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:33.348783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:33.349422 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:39:33.349626 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:33.349631 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.350354 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.350354 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:33.351219 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:33.351222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.351586 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.351586 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:33.351700 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.352235 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:39:33.352244 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:39:33.353740 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:39:33.355681 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:33.355689 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:33.355693 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:33.356037 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:33.356222 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:39:33.356223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:39:33.356223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:39:33.356341 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:39:33.356558 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:39:33.356560 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:39:33.356676 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 14:39:33.356755 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 14:39:33.357152 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260811 14:39:33.357771 2437 heartbeater.cc:347] Connected to a master server at 127.2.103.126:39681
I20260811 14:39:33.357867 2437 heartbeater.cc:464] Registering TS with master...
I20260811 14:39:33.358114 2437 heartbeater.cc:511] Master 127.2.103.126:39681 requested a full tablet report, sending...
I20260811 14:39:33.358515 1635 ts_manager.cc:194] Registered new tserver with Master: 45c5d0d679c34efda229cc99adef9f35 (127.2.103.67:42395)
I20260811 14:39:33.358922 1635 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.103.67@KRBTEST.COM'} at 127.2.103.67:53661
I20260811 14:39:33.367208 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:39:33.374742 1703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.368649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52460 (local address 127.2.103.126:39681)
0811 14:39:33.368718 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:33.368720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:33.368805 (+ 85us) server_negotiation.cc:407] Connection header received
0811 14:39:33.368859 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:33.368859 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:33.368869 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:33.368923 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:33.369442 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.369927 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.370668 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.370858 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.370910 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.372073 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:33.372075 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:33.372075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:33.372082 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:33.373617 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:33.373938 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:33.373939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:33.373940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:33.373993 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:33.374158 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:33.374158 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:33.374158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:33.374309 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:33.374395 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:33.374551 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 14:39:33.374675 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 14:39:33.376467 1635 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52460:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:39:33.378221 1635 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:39:33.385761 2460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.381078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:42929 (local address 127.0.0.1:40150)
0811 14:39:33.381443 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:33.381608 (+ 165us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:33.381661 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:33.381869 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:33.381872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:33.381875 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:33.381963 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:33.381965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.383420 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.383421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:33.384560 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:33.384562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.385442 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.385443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:33.385524 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.385542 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 14:39:33.385621 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 14:39:33.385673 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":41,"thread_start_us":72,"threads_started":1}
I20260811 14:39:33.385777 2458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.381182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40150 (local address 127.2.103.66:42929)
0811 14:39:33.381441 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:33.381446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:33.381659 (+ 213us) server_negotiation.cc:407] Connection header received
0811 14:39:33.381737 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:33.381740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:33.381758 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:33.382040 (+ 282us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:33.382119 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.383309 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.384645 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.385365 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.385434 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.385497 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:33.385642 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 14:39:33.385677 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":121,"threads_started":1}
I20260811 14:39:33.386128 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.381309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:42395 (local address 127.0.0.1:36732)
0811 14:39:33.381604 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:33.381615 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:33.381669 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:33.382037 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:33.382038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:33.382040 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:33.382070 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:33.382071 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.383100 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.383102 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:33.384464 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:33.384467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.385882 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.385884 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:33.385953 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.385957 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:39:33.386017 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 14:39:33.386059 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":31,"thread_start_us":69,"threads_started":1}
I20260811 14:39:33.386363 2468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.381408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36732 (local address 127.2.103.67:42395)
0811 14:39:33.381820 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:33.381826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:33.381836 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:39:33.381885 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:33.381888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:33.381903 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:33.382155 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:33.382236 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.382996 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.384563 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.385353 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.385435 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.385497 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:33.386235 (+ 738us) server_negotiation.cc:299] Negotiation successful
0811 14:39:33.386283 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":93,"threads_started":1}
I20260811 14:39:33.386927 2456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.381195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:46099 (local address 127.0.0.1:39534)
0811 14:39:33.381484 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:33.381498 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:33.381568 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:33.381980 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:33.381981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:33.381984 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:33.382049 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:33.382051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.383423 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.383424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:33.385216 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:33.385218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.386691 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.386693 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:33.386757 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.386761 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:39:33.386816 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 14:39:33.386855 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":998,"thread_start_us":89,"threads_started":1}
I20260811 14:39:33.386935 2088 tablet_service.cc:1467] Processing CreateTablet for tablet 770957af17fc43d89dc16a2382112fc5 (DEFAULT_TABLE table=test-table [id=8fb1579a8e444f6db77751930f082437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:39:33.386988 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.381412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39534 (local address 127.2.103.65:46099)
0811 14:39:33.381814 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:33.381820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:33.381830 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:39:33.381870 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:33.381872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:33.381887 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:33.381965 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:33.382121 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.383292 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.385309 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.386360 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.386558 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.386658 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:33.386851 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 14:39:33.386899 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":95,"threads_started":1}
I20260811 14:39:33.387100 2334 tablet_service.cc:1467] Processing CreateTablet for tablet 770957af17fc43d89dc16a2382112fc5 (DEFAULT_TABLE table=test-table [id=8fb1579a8e444f6db77751930f082437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:39:33.387365 2088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 770957af17fc43d89dc16a2382112fc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:33.387526 2334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 770957af17fc43d89dc16a2382112fc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:33.387683 1848 tablet_service.cc:1467] Processing CreateTablet for tablet 770957af17fc43d89dc16a2382112fc5 (DEFAULT_TABLE table=test-table [id=8fb1579a8e444f6db77751930f082437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:39:33.387996 1848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 770957af17fc43d89dc16a2382112fc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:33.390296 2473 tablet_bootstrap.cc:492] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35: Bootstrap starting.
I20260811 14:39:33.391006 2474 tablet_bootstrap.cc:492] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c: Bootstrap starting.
I20260811 14:39:33.391080 2473 tablet_bootstrap.cc:654] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:33.391698 2473 log.cc:824] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:33.391942 2474 tablet_bootstrap.cc:654] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:33.392047 2476 tablet_bootstrap.cc:492] T 770957af17fc43d89dc16a2382112fc5 P 6beb6ff029cf44fc8f6341bc3be499ba: Bootstrap starting.
I20260811 14:39:33.392434 2474 log.cc:824] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:33.392640 2476 tablet_bootstrap.cc:654] T 770957af17fc43d89dc16a2382112fc5 P 6beb6ff029cf44fc8f6341bc3be499ba: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:33.393136 2474 tablet_bootstrap.cc:492] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c: No bootstrap required, opened a new log
I20260811 14:39:33.393167 2476 log.cc:824] T 770957af17fc43d89dc16a2382112fc5 P 6beb6ff029cf44fc8f6341bc3be499ba: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:33.393226 2474 ts_tablet_manager.cc:1397] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:39:33.393940 2476 tablet_bootstrap.cc:492] T 770957af17fc43d89dc16a2382112fc5 P 6beb6ff029cf44fc8f6341bc3be499ba: No bootstrap required, opened a new log
I20260811 14:39:33.394076 2476 ts_tablet_manager.cc:1397] T 770957af17fc43d89dc16a2382112fc5 P 6beb6ff029cf44fc8f6341bc3be499ba: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:39:33.395033 2473 tablet_bootstrap.cc:492] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35: No bootstrap required, opened a new log
I20260811 14:39:33.395133 2473 ts_tablet_manager.cc:1397] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260811 14:39:33.395303 2474 raft_consensus.cc:348] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f38dd66c97e4c7da71852c360115d2c" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 42929 } } peers { permanent_uuid: "6beb6ff029cf44fc8f6341bc3be499ba" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46099 } } peers { permanent_uuid: "45c5d0d679c34efda229cc99adef9f35" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 42395 } }
I20260811 14:39:33.395511 2474 raft_consensus.cc:374] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:33.395550 2474 raft_consensus.cc:729] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f38dd66c97e4c7da71852c360115d2c, State: Initialized, Role: FOLLOWER
I20260811 14:39:33.395673 2474 consensus_queue.cc:260] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f38dd66c97e4c7da71852c360115d2c" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 42929 } } peers { permanent_uuid: "6beb6ff029cf44fc8f6341bc3be499ba" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46099 } } peers { permanent_uuid: "45c5d0d679c34efda229cc99adef9f35" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 42395 } }
I20260811 14:39:33.395974 2214 heartbeater.cc:503] Master 127.2.103.126:39681 was elected leader, sending a full tablet report...
I20260811 14:39:33.395972 2474 ts_tablet_manager.cc:1428] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:39:33.396153 2476 raft_consensus.cc:348] T 770957af17fc43d89dc16a2382112fc5 P 6beb6ff029cf44fc8f6341bc3be499ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f38dd66c97e4c7da71852c360115d2c" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 42929 } } peers { permanent_uuid: "6beb6ff029cf44fc8f6341bc3be499ba" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46099 } } peers { permanent_uuid: "45c5d0d679c34efda229cc99adef9f35" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 42395 } }
I20260811 14:39:33.396322 2476 raft_consensus.cc:374] T 770957af17fc43d89dc16a2382112fc5 P 6beb6ff029cf44fc8f6341bc3be499ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:33.396370 2476 raft_consensus.cc:729] T 770957af17fc43d89dc16a2382112fc5 P 6beb6ff029cf44fc8f6341bc3be499ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6beb6ff029cf44fc8f6341bc3be499ba, State: Initialized, Role: FOLLOWER
I20260811 14:39:33.396502 2476 consensus_queue.cc:260] T 770957af17fc43d89dc16a2382112fc5 P 6beb6ff029cf44fc8f6341bc3be499ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f38dd66c97e4c7da71852c360115d2c" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 42929 } } peers { permanent_uuid: "6beb6ff029cf44fc8f6341bc3be499ba" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46099 } } peers { permanent_uuid: "45c5d0d679c34efda229cc99adef9f35" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 42395 } }
I20260811 14:39:33.396808 2476 ts_tablet_manager.cc:1428] T 770957af17fc43d89dc16a2382112fc5 P 6beb6ff029cf44fc8f6341bc3be499ba: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:39:33.396808 1976 heartbeater.cc:503] Master 127.2.103.126:39681 was elected leader, sending a full tablet report...
I20260811 14:39:33.397158 2473 raft_consensus.cc:348] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f38dd66c97e4c7da71852c360115d2c" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 42929 } } peers { permanent_uuid: "6beb6ff029cf44fc8f6341bc3be499ba" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46099 } } peers { permanent_uuid: "45c5d0d679c34efda229cc99adef9f35" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 42395 } }
I20260811 14:39:33.397351 2473 raft_consensus.cc:374] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:33.397426 2473 raft_consensus.cc:729] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c5d0d679c34efda229cc99adef9f35, State: Initialized, Role: FOLLOWER
I20260811 14:39:33.397545 2473 consensus_queue.cc:260] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f38dd66c97e4c7da71852c360115d2c" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 42929 } } peers { permanent_uuid: "6beb6ff029cf44fc8f6341bc3be499ba" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46099 } } peers { permanent_uuid: "45c5d0d679c34efda229cc99adef9f35" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 42395 } }
I20260811 14:39:33.397826 2473 ts_tablet_manager.cc:1428] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 14:39:33.397851 2437 heartbeater.cc:503] Master 127.2.103.126:39681 was elected leader, sending a full tablet report...
W20260811 14:39:33.427940 2216 tablet.cc:2369] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:39:33.487291 2487 raft_consensus.cc:482] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:39:33.487449 2487 raft_consensus.cc:504] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f38dd66c97e4c7da71852c360115d2c" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 42929 } } peers { permanent_uuid: "6beb6ff029cf44fc8f6341bc3be499ba" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46099 } } peers { permanent_uuid: "45c5d0d679c34efda229cc99adef9f35" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 42395 } }
I20260811 14:39:33.487915 2487 leader_election.cc:288] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6beb6ff029cf44fc8f6341bc3be499ba (127.2.103.65:46099), 45c5d0d679c34efda229cc99adef9f35 (127.2.103.67:42395)
I20260811 14:39:33.492137 2468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.488276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:60783 (local address 127.2.103.67:42395)
0811 14:39:33.488350 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:33.488353 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:33.488487 (+ 134us) server_negotiation.cc:407] Connection header received
0811 14:39:33.488541 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:33.488542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:33.488551 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:33.488597 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:33.488777 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.489555 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.491171 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.491735 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.491821 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.491937 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:33.492014 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 14:39:33.492060 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 14:39:33.492137 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.488271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:42521 (local address 127.2.103.65:46099)
0811 14:39:33.488343 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:33.488345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:33.488355 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:39:33.488392 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:33.488392 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:33.488405 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:33.488470 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:33.488778 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.489563 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.491172 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.491741 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.491816 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.491826 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:33.492034 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 14:39:33.492069 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 14:39:33.492156 2220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.488134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:46099 (local address 127.2.103.66:42521)
0811 14:39:33.488217 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:33.488229 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:33.488279 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:33.488539 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:33.488540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:33.488542 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:33.488648 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:33.488650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.489650 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.489651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:33.491075 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:33.491079 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.491834 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.491835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:33.491927 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.491936 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 14:39:33.491997 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 14:39:33.492056 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":93}
I20260811 14:39:33.492162 2493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.488172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:42395 (local address 127.2.103.66:60783)
0811 14:39:33.488396 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:33.488405 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:33.488459 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:33.488625 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:33.488626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:33.488628 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:33.488684 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:33.488685 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.489650 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.489651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:33.491075 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:33.491079 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.491854 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.491855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:33.491927 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.491937 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 14:39:33.491997 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 14:39:33.492056 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":61,"thread_start_us":135,"threads_started":1}
I20260811 14:39:33.492635 1886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "770957af17fc43d89dc16a2382112fc5" candidate_uuid: "1f38dd66c97e4c7da71852c360115d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6beb6ff029cf44fc8f6341bc3be499ba" is_pre_election: true
I20260811 14:39:33.492851 1886 raft_consensus.cc:2454] T 770957af17fc43d89dc16a2382112fc5 P 6beb6ff029cf44fc8f6341bc3be499ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f38dd66c97e4c7da71852c360115d2c in term 0.
I20260811 14:39:33.492849 2361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "770957af17fc43d89dc16a2382112fc5" candidate_uuid: "1f38dd66c97e4c7da71852c360115d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c5d0d679c34efda229cc99adef9f35" is_pre_election: true
I20260811 14:39:33.492995 2361 raft_consensus.cc:2454] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f38dd66c97e4c7da71852c360115d2c in term 0.
I20260811 14:39:33.493122 2009 leader_election.cc:302] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f38dd66c97e4c7da71852c360115d2c, 6beb6ff029cf44fc8f6341bc3be499ba; no voters:
I20260811 14:39:33.493264 2487 raft_consensus.cc:2781] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:39:33.493322 2487 raft_consensus.cc:482] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:39:33.493350 2487 raft_consensus.cc:3037] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:33.494131 2487 raft_consensus.cc:504] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f38dd66c97e4c7da71852c360115d2c" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 42929 } } peers { permanent_uuid: "6beb6ff029cf44fc8f6341bc3be499ba" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46099 } } peers { permanent_uuid: "45c5d0d679c34efda229cc99adef9f35" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 42395 } }
I20260811 14:39:33.494311 2487 leader_election.cc:288] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [CANDIDATE]: Term 1 election: Requested vote from peers 6beb6ff029cf44fc8f6341bc3be499ba (127.2.103.65:46099), 45c5d0d679c34efda229cc99adef9f35 (127.2.103.67:42395)
I20260811 14:39:33.494477 1886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "770957af17fc43d89dc16a2382112fc5" candidate_uuid: "1f38dd66c97e4c7da71852c360115d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6beb6ff029cf44fc8f6341bc3be499ba"
I20260811 14:39:33.494481 2361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "770957af17fc43d89dc16a2382112fc5" candidate_uuid: "1f38dd66c97e4c7da71852c360115d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c5d0d679c34efda229cc99adef9f35"
I20260811 14:39:33.494555 1886 raft_consensus.cc:3037] T 770957af17fc43d89dc16a2382112fc5 P 6beb6ff029cf44fc8f6341bc3be499ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:33.494591 2361 raft_consensus.cc:3037] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:33.495378 1886 raft_consensus.cc:2454] T 770957af17fc43d89dc16a2382112fc5 P 6beb6ff029cf44fc8f6341bc3be499ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f38dd66c97e4c7da71852c360115d2c in term 1.
I20260811 14:39:33.495523 2361 raft_consensus.cc:2454] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f38dd66c97e4c7da71852c360115d2c in term 1.
I20260811 14:39:33.495617 2009 leader_election.cc:302] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f38dd66c97e4c7da71852c360115d2c, 6beb6ff029cf44fc8f6341bc3be499ba; no voters:
I20260811 14:39:33.495746 2487 raft_consensus.cc:2781] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:33.495930 2487 raft_consensus.cc:686] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [term 1 LEADER]: Becoming Leader. State: Replica: 1f38dd66c97e4c7da71852c360115d2c, State: Running, Role: LEADER
I20260811 14:39:33.496042 2487 consensus_queue.cc:237] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f38dd66c97e4c7da71852c360115d2c" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 42929 } } peers { permanent_uuid: "6beb6ff029cf44fc8f6341bc3be499ba" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46099 } } peers { permanent_uuid: "45c5d0d679c34efda229cc99adef9f35" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 42395 } }
I20260811 14:39:33.496946 1635 catalog_manager.cc:5495] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f38dd66c97e4c7da71852c360115d2c (127.2.103.66). New cstate: current_term: 1 leader_uuid: "1f38dd66c97e4c7da71852c360115d2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f38dd66c97e4c7da71852c360115d2c" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 42929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6beb6ff029cf44fc8f6341bc3be499ba" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45c5d0d679c34efda229cc99adef9f35" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 42395 } health_report { overall_health: UNKNOWN } } }
I20260811 14:39:33.502847 2458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.500563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40154 (local address 127.2.103.66:42929)
0811 14:39:33.500625 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:33.500627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:33.500635 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:39:33.500663 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:33.500663 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:33.500680 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:33.500740 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:39:33.500848 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.501286 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.502008 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.502188 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.502240 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.502436 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:39:33.502529 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:39:33.502595 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:33.502703 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 14:39:33.502729 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 14:39:33.504616 2361 raft_consensus.cc:1261] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35 [term 1 FOLLOWER]: Refusing update from remote peer 1f38dd66c97e4c7da71852c360115d2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:39:33.504632 1886 raft_consensus.cc:1261] T 770957af17fc43d89dc16a2382112fc5 P 6beb6ff029cf44fc8f6341bc3be499ba [term 1 FOLLOWER]: Refusing update from remote peer 1f38dd66c97e4c7da71852c360115d2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:39:33.504961 2494 consensus_queue.cc:1035] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "45c5d0d679c34efda229cc99adef9f35" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 42395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:39:33.505095 2487 consensus_queue.cc:1035] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6beb6ff029cf44fc8f6341bc3be499ba" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 46099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:39:33.507396 2501 mvcc.cc:205] Tried to move back new op lower bound from 7317336774672158720 to 7317336774640427008. Current Snapshot: MvccSnapshot[applied={T|T < 7317336774672158720}]
I20260811 14:39:33.507911 2503 mvcc.cc:205] Tried to move back new op lower bound from 7317336774672158720 to 7317336774640427008. Current Snapshot: MvccSnapshot[applied={T|T < 7317336774672158720}]
I20260811 14:39:33.510486 1631 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:39:33.510577 1631 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:39:33.511438 1631 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 07b8dcec27fd465cb8e795b6afb324d7: Sending DeleteTablet for 3 replicas of tablet 770957af17fc43d89dc16a2382112fc5
I20260811 14:39:33.511797 1848 tablet_service.cc:1514] Processing DeleteTablet for tablet 770957af17fc43d89dc16a2382112fc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:39:33 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:39534
I20260811 14:39:33.511822 2088 tablet_service.cc:1514] Processing DeleteTablet for tablet 770957af17fc43d89dc16a2382112fc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:39:33 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:40150
I20260811 14:39:33.511929 2334 tablet_service.cc:1514] Processing DeleteTablet for tablet 770957af17fc43d89dc16a2382112fc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:39:33 UTC) from {username='kudu', principal='kudu/127.2.103.126@KRBTEST.COM'} at 127.0.0.1:36732
I20260811 14:39:33.512061 2513 tablet_replica.cc:330] T 770957af17fc43d89dc16a2382112fc5 P 6beb6ff029cf44fc8f6341bc3be499ba: stopping tablet replica
I20260811 14:39:33.512161 2513 raft_consensus.cc:2229] T 770957af17fc43d89dc16a2382112fc5 P 6beb6ff029cf44fc8f6341bc3be499ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:39:33.512279 2513 raft_consensus.cc:2258] T 770957af17fc43d89dc16a2382112fc5 P 6beb6ff029cf44fc8f6341bc3be499ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:39:33.512343 2515 tablet_replica.cc:330] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c: stopping tablet replica
I20260811 14:39:33.512347 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 1742
I20260811 14:39:33.512439 2515 raft_consensus.cc:2229] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:39:33.512547 2515 raft_consensus.cc:2258] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:39:33.512671 2517 tablet_replica.cc:330] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35: stopping tablet replica
I20260811 14:39:33.512765 2517 raft_consensus.cc:2229] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:39:33.512862 2517 raft_consensus.cc:2258] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:39:33.515642 2515 ts_tablet_manager.cc:1905] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:39:33.517469 2515 ts_tablet_manager.cc:1918] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:39:33.517525 2515 log.cc:1196] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/770957af17fc43d89dc16a2382112fc5
I20260811 14:39:33.517728 2515 ts_tablet_manager.cc:1939] T 770957af17fc43d89dc16a2382112fc5 P 1f38dd66c97e4c7da71852c360115d2c: Deleting consensus metadata
I20260811 14:39:33.518278 1612 catalog_manager.cc:4841] TS 1f38dd66c97e4c7da71852c360115d2c (127.2.103.66:42929): tablet 770957af17fc43d89dc16a2382112fc5 (table test-table [id=8fb1579a8e444f6db77751930f082437]) successfully deleted
I20260811 14:39:33.542840 2517 ts_tablet_manager.cc:1905] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:39:33.544832 2517 ts_tablet_manager.cc:1918] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:39:33.544941 2517 log.cc:1196] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/770957af17fc43d89dc16a2382112fc5
I20260811 14:39:33.545246 2517 ts_tablet_manager.cc:1939] T 770957af17fc43d89dc16a2382112fc5 P 45c5d0d679c34efda229cc99adef9f35: Deleting consensus metadata
I20260811 14:39:33.545807 1612 catalog_manager.cc:4841] TS 45c5d0d679c34efda229cc99adef9f35 (127.2.103.67:42395): tablet 770957af17fc43d89dc16a2382112fc5 (table test-table [id=8fb1579a8e444f6db77751930f082437]) successfully deleted
W20260811 14:39:33.547186 1615 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.103.65:46099 (error 108)
I20260811 14:39:33.547827 2456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.547670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:46099 (local address 127.0.0.1:39540)
0811 14:39:33.547734 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:33.547790 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.65:46099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 14:39:33.547895 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 1985
W20260811 14:39:33.547957 1615 catalog_manager.cc:4584] TS 6beb6ff029cf44fc8f6341bc3be499ba (127.2.103.65:46099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 770957af17fc43d89dc16a2382112fc5: Network error: Client connection negotiation failed: client connection to 127.2.103.65:46099: connect: Connection refused (error 111)
I20260811 14:39:33.564069 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 2226
I20260811 14:39:33.577808 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 1583
2026-08-11T14:39:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2370 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T14:39:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:33Z Disabled control of system clock
I20260811 14:39:33.613444 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:38583
--webserver_interface=127.2.103.126
--webserver_port=0
--builtin_ntp_servers=127.2.103.84:45901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:38583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 14:39:33.699026 2524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:33.699254 2524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:33.699276 2524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:33.700459 2524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:33.700502 2524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:33.700516 2524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:33.700528 2524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:33.700541 2524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:33.701915 2524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:45901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:38583
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:38583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:33.702190 2524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:33.702508 2524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:33.707501 2533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:33.707553 2529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:33.707580 2531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:33.707901 2524 server_base.cc:1034] running on GCE node
I20260811 14:39:33.708194 2524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:33.708463 2524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:33.709636 2524 hybrid_clock.cc:648] HybridClock initialized: now 1786459173709629 us; error 37 us; skew 500 ppm
I20260811 14:39:33.711019 2524 webserver.cc:460] Webserver started at http://127.2.103.126:41577/ using document root <none> and password file <none>
I20260811 14:39:33.711280 2524 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:33.711350 2524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:33.711511 2524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:33.712600 2524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "fb3a94ba03a74ce0ae3888047bde2324"
format_stamp: "Formatted at 2026-08-11 14:39:33 on dist-test-slave-twwt"
server_key: "81449e58bf6657204aa9d7a73c04d498"
server_key_iv: "aa297581b9e68c368a8553f2c6f646c2"
server_key_version: "encryptionkey@0"
I20260811 14:39:33.713004 2524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "fb3a94ba03a74ce0ae3888047bde2324"
format_stamp: "Formatted at 2026-08-11 14:39:33 on dist-test-slave-twwt"
server_key: "81449e58bf6657204aa9d7a73c04d498"
server_key_iv: "aa297581b9e68c368a8553f2c6f646c2"
server_key_version: "encryptionkey@0"
I20260811 14:39:33.714368 2524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:39:33.715207 2549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:33.715379 2524 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:33.715487 2524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "fb3a94ba03a74ce0ae3888047bde2324"
format_stamp: "Formatted at 2026-08-11 14:39:33 on dist-test-slave-twwt"
server_key: "81449e58bf6657204aa9d7a73c04d498"
server_key_iv: "aa297581b9e68c368a8553f2c6f646c2"
server_key_version: "encryptionkey@0"
I20260811 14:39:33.715570 2524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:33.726305 2524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:33.726754 2524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:33.726928 2524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:33.731089 2524 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:38583
I20260811 14:39:33.731127 2641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:38583 every 8 connection(s)
I20260811 14:39:33.731490 2524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:33.732540 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:33.735054 2647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324: Bootstrap starting.
I20260811 14:39:33.735695 2647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:33.736040 2647 log.cc:824] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:33.736792 2647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324: No bootstrap required, opened a new log
I20260811 14:39:33.738508 2647 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3a94ba03a74ce0ae3888047bde2324" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38583 } }
I20260811 14:39:33.738747 2647 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:33.738811 2647 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb3a94ba03a74ce0ae3888047bde2324, State: Initialized, Role: FOLLOWER
I20260811 14:39:33.738896 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 2524
I20260811 14:39:33.739051 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:33.739004 2647 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3a94ba03a74ce0ae3888047bde2324" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38583 } }
I20260811 14:39:33.739096 2647 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:33.739147 2647 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:33.739202 2647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:33.739282 2461 external_mini_cluster.cc:1428] Setting key ab6eb472954c7d0a6083fd8d162efeb2
I20260811 14:39:33.740044 2647 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3a94ba03a74ce0ae3888047bde2324" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38583 } }
I20260811 14:39:33.740167 2647 leader_election.cc:302] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb3a94ba03a74ce0ae3888047bde2324; no voters:
I20260811 14:39:33.740335 2647 leader_election.cc:288] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:33.740412 2653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:33.740588 2653 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324 [term 1 LEADER]: Becoming Leader. State: Replica: fb3a94ba03a74ce0ae3888047bde2324, State: Running, Role: LEADER
I20260811 14:39:33.740614 2647 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:33.740679 2653 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3a94ba03a74ce0ae3888047bde2324" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38583 } }
I20260811 14:39:33.741061 2653 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb3a94ba03a74ce0ae3888047bde2324. Latest consensus state: current_term: 1 leader_uuid: "fb3a94ba03a74ce0ae3888047bde2324" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3a94ba03a74ce0ae3888047bde2324" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38583 } } }
I20260811 14:39:33.741155 2653 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:33.741365 2666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:33.741374 2655 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb3a94ba03a74ce0ae3888047bde2324" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3a94ba03a74ce0ae3888047bde2324" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 38583 } } }
I20260811 14:39:33.741581 2655 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:33.741938 2666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:33.744176 2666 catalog_manager.cc:1347] Generated new cluster ID: e9abb6c4942b4e5eae845ebe3271eb7e
I20260811 14:39:33.744241 2666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:39:33.745891 2652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.739892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46288 (local address 127.2.103.126:38583)
0811 14:39:33.740171 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:33.740181 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:33.740217 (+ 36us) server_negotiation.cc:407] Connection header received
0811 14:39:33.740360 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:33.740368 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:33.740457 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:33.740879 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:33.740979 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.742532 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.743618 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.744193 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.744667 (+ 474us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.745119 (+ 452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:33.745124 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:33.745124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:33.745355 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:33.745409 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:33.745425 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:33.745503 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:33.745577 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 14:39:33.745664 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":115,"threads_started":1}
I20260811 14:39:33.752969 2666 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:33.753568 2666 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:33.755978 2666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324: Generated new TSK 0
I20260811 14:39:33.756204 2666 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:39:33.759320 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.65:0
--local_ip_for_outbound_sockets=127.2.103.65
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38583
--builtin_ntp_servers=127.2.103.84:45901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 14:39:33.849009 2683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:33.849268 2683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:33.849308 2683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:33.850530 2683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:33.850602 2683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:33.850679 2683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.65
I20260811 14:39:33.852178 2683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:45901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.103.65
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.65
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:33.852473 2683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:33.852830 2683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:33.858877 2693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:33.858889 2694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:33.859002 2697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:33.859264 2683 server_base.cc:1034] running on GCE node
I20260811 14:39:33.859485 2683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:33.859751 2683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:33.860924 2683 hybrid_clock.cc:648] HybridClock initialized: now 1786459173860908 us; error 28 us; skew 500 ppm
I20260811 14:39:33.862507 2683 webserver.cc:460] Webserver started at http://127.2.103.65:33605/ using document root <none> and password file <none>
I20260811 14:39:33.862833 2683 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:33.862897 2683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:33.863035 2683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:33.864540 2683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-0/data/instance:
uuid: "7451935e26554c93876f66657fe47621"
format_stamp: "Formatted at 2026-08-11 14:39:33 on dist-test-slave-twwt"
server_key: "c73002741c9b922c140e3980f5ed3383"
server_key_iv: "c3732700910e0530312f809393d13868"
server_key_version: "encryptionkey@0"
I20260811 14:39:33.865056 2683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance:
uuid: "7451935e26554c93876f66657fe47621"
format_stamp: "Formatted at 2026-08-11 14:39:33 on dist-test-slave-twwt"
server_key: "c73002741c9b922c140e3980f5ed3383"
server_key_iv: "c3732700910e0530312f809393d13868"
server_key_version: "encryptionkey@0"
I20260811 14:39:33.866966 2683 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 14:39:33.867887 2705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:33.868100 2683 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:33.868194 2683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-0/wal
uuid: "7451935e26554c93876f66657fe47621"
format_stamp: "Formatted at 2026-08-11 14:39:33 on dist-test-slave-twwt"
server_key: "c73002741c9b922c140e3980f5ed3383"
server_key_iv: "c3732700910e0530312f809393d13868"
server_key_version: "encryptionkey@0"
I20260811 14:39:33.868260 2683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:33.890172 2683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:33.890508 2683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:33.890651 2683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:33.890880 2683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:33.891280 2683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:39:33.891316 2683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:33.891379 2683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:39:33.891405 2683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:33.897735 2683 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.65:35629
I20260811 14:39:33.897804 2903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.65:35629 every 8 connection(s)
I20260811 14:39:33.898159 2683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-0/data/info.pb
I20260811 14:39:33.903939 2652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.898962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:55561 (local address 127.2.103.126:38583)
0811 14:39:33.899016 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:33.899023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:33.899248 (+ 225us) server_negotiation.cc:407] Connection header received
0811 14:39:33.899821 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:33.899822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:33.899839 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:33.899901 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:33.900622 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.901134 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.902221 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.902450 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.902526 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.903463 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:33.903465 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:33.903465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:33.903473 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:33.903496 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:33.903502 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:33.903557 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:33.903738 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 14:39:33.903816 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:39:33.903991 2909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:33.898964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:38583 (local address 127.2.103.65:55561)
0811 14:39:33.899136 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:33.899172 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:33.899669 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:33.899960 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:33.899964 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:33.899973 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:39:33.900514 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:33.900518 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.901229 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.901230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:33.902110 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:33.902113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:33.902543 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:33.902544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:33.902680 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:33.903250 (+ 570us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:39:33.903250 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:39:33.903291 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:39:33.903316 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:39:33.903609 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:39:33.903617 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:39:33.903620 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:39:33.903708 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 14:39:33.903801 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":74,"threads_started":1}
I20260811 14:39:33.904639 2905 heartbeater.cc:347] Connected to a master server at 127.2.103.126:38583
I20260811 14:39:33.904762 2905 heartbeater.cc:464] Registering TS with master...
I20260811 14:39:33.905000 2905 heartbeater.cc:511] Master 127.2.103.126:38583 requested a full tablet report, sending...
I20260811 14:39:33.905560 2572 ts_manager.cc:194] Registered new tserver with Master: 7451935e26554c93876f66657fe47621 (127.2.103.65:35629)
I20260811 14:39:33.906226 2572 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.103.65:55561
I20260811 14:39:33.906558 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 2683
I20260811 14:39:33.906705 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-0/wal/instance
I20260811 14:39:33.906926 2461 external_mini_cluster.cc:1428] Setting key ed1a285e36b1b8063e2413aadfc719a9
I20260811 14:39:33.907868 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.66:0
--local_ip_for_outbound_sockets=127.2.103.66
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38583
--builtin_ntp_servers=127.2.103.84:45901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 14:39:33.997947 2910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:33.998174 2910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:33.998198 2910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:33.999426 2910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:33.999471 2910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:33.999497 2910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.66
I20260811 14:39:34.000793 2910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:45901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.103.66
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.66
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:34.001003 2910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:34.001271 2910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:34.006547 2917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:34.006517 2916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:34.006536 2921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:34.006785 2910 server_base.cc:1034] running on GCE node
I20260811 14:39:34.007023 2910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:34.007251 2910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:34.008424 2910 hybrid_clock.cc:648] HybridClock initialized: now 1786459174008410 us; error 35 us; skew 500 ppm
I20260811 14:39:34.009883 2910 webserver.cc:460] Webserver started at http://127.2.103.66:44111/ using document root <none> and password file <none>
I20260811 14:39:34.010161 2910 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:34.010239 2910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:34.010401 2910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:34.011628 2910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-1/data/instance:
uuid: "28a516f0f93d4c72b9e140b57bcfed7f"
format_stamp: "Formatted at 2026-08-11 14:39:34 on dist-test-slave-twwt"
server_key: "ec7333601c1d6ef0cc1dc9cdf5030cdf"
server_key_iv: "473aeeda5c1e7b378a2c6a5005cea8d1"
server_key_version: "encryptionkey@0"
I20260811 14:39:34.012024 2910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance:
uuid: "28a516f0f93d4c72b9e140b57bcfed7f"
format_stamp: "Formatted at 2026-08-11 14:39:34 on dist-test-slave-twwt"
server_key: "ec7333601c1d6ef0cc1dc9cdf5030cdf"
server_key_iv: "473aeeda5c1e7b378a2c6a5005cea8d1"
server_key_version: "encryptionkey@0"
I20260811 14:39:34.013432 2910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:39:34.014190 2930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:34.014377 2910 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:39:34.014490 2910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-1/wal
uuid: "28a516f0f93d4c72b9e140b57bcfed7f"
format_stamp: "Formatted at 2026-08-11 14:39:34 on dist-test-slave-twwt"
server_key: "ec7333601c1d6ef0cc1dc9cdf5030cdf"
server_key_iv: "473aeeda5c1e7b378a2c6a5005cea8d1"
server_key_version: "encryptionkey@0"
I20260811 14:39:34.014570 2910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:34.032469 2910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:34.032809 2910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:34.032930 2910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:34.033164 2910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:34.033571 2910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:39:34.033610 2910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:34.033634 2910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:39:34.033687 2910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:34.039743 2910 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.66:37795
I20260811 14:39:34.039860 3138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.66:37795 every 8 connection(s)
I20260811 14:39:34.040236 2910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-1/data/info.pb
I20260811 14:39:34.044099 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 2910
I20260811 14:39:34.044250 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-1/wal/instance
I20260811 14:39:34.044477 2461 external_mini_cluster.cc:1428] Setting key c659194a363744dae637e3e7df2926f5
I20260811 14:39:34.045501 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.103.67:0
--local_ip_for_outbound_sockets=127.2.103.67
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38583
--builtin_ntp_servers=127.2.103.84:45901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 14:39:34.046346 2652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:34.041156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.66:39301 (local address 127.2.103.126:38583)
0811 14:39:34.041211 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:34.041213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:34.041498 (+ 285us) server_negotiation.cc:407] Connection header received
0811 14:39:34.042136 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:34.042138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:34.042152 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:34.042204 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:34.043055 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.043570 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.044730 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.044982 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.045066 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:34.046042 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:34.046043 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:34.046044 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:34.046048 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:34.046064 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:34.046068 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:34.046120 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:34.046233 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 14:39:34.046265 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:39:34.046597 3144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:34.041134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:38583 (local address 127.2.103.66:39301)
0811 14:39:34.041369 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:34.041406 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:34.042008 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:34.042303 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:34.042307 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:34.042325 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:39:34.042946 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:34.042952 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.043673 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.043674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:34.044640 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:34.044643 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.045078 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.045079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:34.045229 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:34.045869 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:39:34.045870 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:39:34.045902 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:39:34.045927 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:39:34.046134 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:39:34.046139 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:39:34.046142 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:39:34.046223 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 14:39:34.046339 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":99,"threads_started":1}
I20260811 14:39:34.047183 3142 heartbeater.cc:347] Connected to a master server at 127.2.103.126:38583
I20260811 14:39:34.047310 3142 heartbeater.cc:464] Registering TS with master...
I20260811 14:39:34.047550 3142 heartbeater.cc:511] Master 127.2.103.126:38583 requested a full tablet report, sending...
I20260811 14:39:34.048063 2572 ts_manager.cc:194] Registered new tserver with Master: 28a516f0f93d4c72b9e140b57bcfed7f (127.2.103.66:37795)
I20260811 14:39:34.048584 2572 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.103.66:39301
W20260811 14:39:34.138600 3146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:34.138929 3146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:34.138975 3146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:34.140209 3146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:34.140295 3146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:34.140337 3146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.103.67
I20260811 14:39:34.142001 3146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:45901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.103.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.103.67
--webserver_port=0
--tserver_master_addrs=127.2.103.126:38583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.103.67
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:34.142333 3146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:34.142728 3146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:34.148068 3160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:34.148267 3146 server_base.cc:1034] running on GCE node
W20260811 14:39:34.148331 3157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:34.148130 3155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:34.148660 3146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:34.148936 3146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:34.150120 3146 hybrid_clock.cc:648] HybridClock initialized: now 1786459174150112 us; error 35 us; skew 500 ppm
I20260811 14:39:34.151530 3146 webserver.cc:460] Webserver started at http://127.2.103.67:33005/ using document root <none> and password file <none>
I20260811 14:39:34.151836 3146 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:34.151917 3146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:34.152078 3146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:34.153225 3146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-2/data/instance:
uuid: "28c1866178d846a58d02d9b5c3b31d4d"
format_stamp: "Formatted at 2026-08-11 14:39:34 on dist-test-slave-twwt"
server_key: "25ab55d3dae8af4b9ab7889f0185932e"
server_key_iv: "69c4e7335d68ffeaee7142436d546a5b"
server_key_version: "encryptionkey@0"
I20260811 14:39:34.153622 3146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance:
uuid: "28c1866178d846a58d02d9b5c3b31d4d"
format_stamp: "Formatted at 2026-08-11 14:39:34 on dist-test-slave-twwt"
server_key: "25ab55d3dae8af4b9ab7889f0185932e"
server_key_iv: "69c4e7335d68ffeaee7142436d546a5b"
server_key_version: "encryptionkey@0"
I20260811 14:39:34.155069 3146 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:39:34.155885 3172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:34.156065 3146 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:34.156179 3146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-2/wal
uuid: "28c1866178d846a58d02d9b5c3b31d4d"
format_stamp: "Formatted at 2026-08-11 14:39:34 on dist-test-slave-twwt"
server_key: "25ab55d3dae8af4b9ab7889f0185932e"
server_key_iv: "69c4e7335d68ffeaee7142436d546a5b"
server_key_version: "encryptionkey@0"
I20260811 14:39:34.156257 3146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:34.178198 3146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:34.178663 3146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:34.178845 3146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:34.179134 3146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:39:34.179574 3146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:39:34.179651 3146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:34.179689 3146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:39:34.179734 3146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:34.185956 3146 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.67:46767
I20260811 14:39:34.186012 3367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.67:46767 every 8 connection(s)
I20260811 14:39:34.186388 3146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-2/data/info.pb
I20260811 14:39:34.192195 2652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:34.187223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.67:34593 (local address 127.2.103.126:38583)
0811 14:39:34.187291 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:34.187293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:34.187566 (+ 273us) server_negotiation.cc:407] Connection header received
0811 14:39:34.188126 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:34.188128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:34.188141 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:34.188210 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:34.188948 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.189473 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.190504 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.190790 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.190887 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:34.191846 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:34.191847 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:34.191848 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:34.191853 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:34.191878 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:34.191883 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:34.191960 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:34.192072 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 14:39:34.192108 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 14:39:34.192448 3373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:34.187211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:38583 (local address 127.2.103.67:34593)
0811 14:39:34.187437 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:34.187472 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:34.188013 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:34.188293 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:34.188297 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:34.188305 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:39:34.188846 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:34.188851 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.189579 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.189581 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:34.190401 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:34.190404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.190903 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.190904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:34.191052 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:34.191681 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:39:34.191681 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:39:34.191715 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:39:34.191741 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:39:34.191960 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:39:34.191967 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:39:34.191971 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:39:34.192058 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 14:39:34.192172 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":102,"threads_started":1}
I20260811 14:39:34.193096 3370 heartbeater.cc:347] Connected to a master server at 127.2.103.126:38583
I20260811 14:39:34.193212 3370 heartbeater.cc:464] Registering TS with master...
I20260811 14:39:34.193372 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 3146
I20260811 14:39:34.193424 3370 heartbeater.cc:511] Master 127.2.103.126:38583 requested a full tablet report, sending...
I20260811 14:39:34.193503 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-2/wal/instance
I20260811 14:39:34.193735 2461 external_mini_cluster.cc:1428] Setting key 0f817ff9f0c28561b09da2b52bafb904
I20260811 14:39:34.193863 2572 ts_manager.cc:194] Registered new tserver with Master: 28c1866178d846a58d02d9b5c3b31d4d (127.2.103.67:46767)
I20260811 14:39:34.194381 2572 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.103.67:34593
I20260811 14:39:34.194700 2461 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:39:34.199316 2652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:34.196209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46296 (local address 127.2.103.126:38583)
0811 14:39:34.196255 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:34.196257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:34.196677 (+ 420us) server_negotiation.cc:407] Connection header received
0811 14:39:34.196720 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:34.196721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:34.196732 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:34.196782 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:34.196920 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.197393 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.198117 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.198297 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.198356 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:34.199100 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:34.199101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:34.199101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:34.199106 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:34.199118 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:34.199122 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:34.199168 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:34.199210 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 14:39:34.199245 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:39:34.201298 2571 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:39:34.202982 2571 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:39:34.209628 3389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:34.205615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:35629 (local address 127.0.0.1:50390)
0811 14:39:34.205866 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:34.205876 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:34.205943 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:34.206267 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:34.206270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:34.206275 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:34.206369 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:34.206372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.207848 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.207849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:34.208588 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:34.208590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.209311 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.209312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:34.209389 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:34.209405 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 14:39:34.209485 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 14:39:34.209543 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":53,"thread_start_us":96,"threads_started":1}
I20260811 14:39:34.209640 3392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:34.205852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50390 (local address 127.2.103.65:35629)
0811 14:39:34.206066 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:34.206070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:34.206080 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:39:34.206117 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:34.206119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:34.206135 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:34.206203 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:34.206463 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.207674 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.208685 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.209200 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.209304 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:34.209418 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:34.209500 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 14:39:34.209543 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":82,"threads_started":1}
I20260811 14:39:34.209825 3388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:34.205507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58230 (local address 127.2.103.66:37795)
0811 14:39:34.205768 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:34.205774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:34.206032 (+ 258us) server_negotiation.cc:407] Connection header received
0811 14:39:34.206086 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:34.206089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:34.206106 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:34.206180 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:34.206446 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.207281 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.208711 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.209471 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.209558 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:34.209649 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:34.209689 (+ 40us) server_negotiation.cc:299] Negotiation successful
0811 14:39:34.209729 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":99,"threads_started":1}
I20260811 14:39:34.210076 3385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:34.205386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:37795 (local address 127.0.0.1:58230)
0811 14:39:34.205812 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:34.205859 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:34.205942 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:34.206267 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:34.206270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:34.206272 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:34.206369 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:34.206373 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.207386 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.207387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:34.208610 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:34.208613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.209558 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.209560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:34.209630 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:34.209633 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:39:34.209675 (+ 42us) client_negotiation.cc:240] Negotiation successful
0811 14:39:34.209720 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260811 14:39:34.210630 2791 tablet_service.cc:1467] Processing CreateTablet for tablet b7025dfb4210492f8aa5bce8c0e9fd73 (DEFAULT_TABLE table=test-table [id=2b3dee080ac0432fac1445ebcf5b2e8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:39:34.210978 3387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:34.205497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:46767 (local address 127.0.0.1:39220)
0811 14:39:34.205837 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:34.205875 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:34.205943 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:34.206417 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:34.206418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:34.206420 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:34.206468 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:34.206469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.207835 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.207836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:34.209251 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:34.209254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.210718 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.210719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:34.210791 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:34.210796 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:39:34.210854 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 14:39:34.210898 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":30,"thread_start_us":113,"threads_started":1}
I20260811 14:39:34.211078 2791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7025dfb4210492f8aa5bce8c0e9fd73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:34.210994 3028 tablet_service.cc:1467] Processing CreateTablet for tablet b7025dfb4210492f8aa5bce8c0e9fd73 (DEFAULT_TABLE table=test-table [id=2b3dee080ac0432fac1445ebcf5b2e8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:39:34.211246 3391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:34.205595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39220 (local address 127.2.103.67:46767)
0811 14:39:34.205816 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:34.205820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:34.206049 (+ 229us) server_negotiation.cc:407] Connection header received
0811 14:39:34.206098 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:34.206100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:34.206117 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:34.206335 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:34.206558 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.207676 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.209852 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.210598 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.210716 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:34.210809 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:34.211103 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 14:39:34.211154 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":108,"threads_started":1}
I20260811 14:39:34.211380 3028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7025dfb4210492f8aa5bce8c0e9fd73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:34.211952 3245 tablet_service.cc:1467] Processing CreateTablet for tablet b7025dfb4210492f8aa5bce8c0e9fd73 (DEFAULT_TABLE table=test-table [id=2b3dee080ac0432fac1445ebcf5b2e8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:39:34.212282 3245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7025dfb4210492f8aa5bce8c0e9fd73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:34.213853 3393 tablet_bootstrap.cc:492] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621: Bootstrap starting.
I20260811 14:39:34.214581 3393 tablet_bootstrap.cc:654] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:34.214973 3393 log.cc:824] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:34.215211 3394 tablet_bootstrap.cc:492] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f: Bootstrap starting.
I20260811 14:39:34.215840 3393 tablet_bootstrap.cc:492] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621: No bootstrap required, opened a new log
I20260811 14:39:34.215947 3394 tablet_bootstrap.cc:654] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:34.216023 3398 tablet_bootstrap.cc:492] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d: Bootstrap starting.
I20260811 14:39:34.216120 3393 ts_tablet_manager.cc:1397] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 14:39:34.216295 3394 log.cc:824] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:34.216856 3398 tablet_bootstrap.cc:654] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:34.217168 3394 tablet_bootstrap.cc:492] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f: No bootstrap required, opened a new log
I20260811 14:39:34.217262 3394 ts_tablet_manager.cc:1397] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:39:34.217291 3398 log.cc:824] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:34.218933 3393 raft_consensus.cc:348] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a516f0f93d4c72b9e140b57bcfed7f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 37795 } } peers { permanent_uuid: "7451935e26554c93876f66657fe47621" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35629 } } peers { permanent_uuid: "28c1866178d846a58d02d9b5c3b31d4d" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46767 } }
I20260811 14:39:34.219156 3393 raft_consensus.cc:374] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:34.219204 3393 raft_consensus.cc:729] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7451935e26554c93876f66657fe47621, State: Initialized, Role: FOLLOWER
I20260811 14:39:34.219296 3398 tablet_bootstrap.cc:492] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d: No bootstrap required, opened a new log
I20260811 14:39:34.219349 3393 consensus_queue.cc:260] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a516f0f93d4c72b9e140b57bcfed7f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 37795 } } peers { permanent_uuid: "7451935e26554c93876f66657fe47621" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35629 } } peers { permanent_uuid: "28c1866178d846a58d02d9b5c3b31d4d" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46767 } }
I20260811 14:39:34.219383 3398 ts_tablet_manager.cc:1397] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 14:39:34.219663 2905 heartbeater.cc:503] Master 127.2.103.126:38583 was elected leader, sending a full tablet report...
I20260811 14:39:34.219650 3393 ts_tablet_manager.cc:1428] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260811 14:39:34.220134 3394 raft_consensus.cc:348] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a516f0f93d4c72b9e140b57bcfed7f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 37795 } } peers { permanent_uuid: "7451935e26554c93876f66657fe47621" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35629 } } peers { permanent_uuid: "28c1866178d846a58d02d9b5c3b31d4d" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46767 } }
I20260811 14:39:34.220350 3394 raft_consensus.cc:374] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:34.220402 3394 raft_consensus.cc:729] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28a516f0f93d4c72b9e140b57bcfed7f, State: Initialized, Role: FOLLOWER
I20260811 14:39:34.220535 3394 consensus_queue.cc:260] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a516f0f93d4c72b9e140b57bcfed7f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 37795 } } peers { permanent_uuid: "7451935e26554c93876f66657fe47621" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35629 } } peers { permanent_uuid: "28c1866178d846a58d02d9b5c3b31d4d" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46767 } }
I20260811 14:39:34.220798 3394 ts_tablet_manager.cc:1428] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260811 14:39:34.220841 3142 heartbeater.cc:503] Master 127.2.103.126:38583 was elected leader, sending a full tablet report...
I20260811 14:39:34.221995 3398 raft_consensus.cc:348] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a516f0f93d4c72b9e140b57bcfed7f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 37795 } } peers { permanent_uuid: "7451935e26554c93876f66657fe47621" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35629 } } peers { permanent_uuid: "28c1866178d846a58d02d9b5c3b31d4d" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46767 } }
I20260811 14:39:34.222128 3398 raft_consensus.cc:374] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:34.222151 3398 raft_consensus.cc:729] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28c1866178d846a58d02d9b5c3b31d4d, State: Initialized, Role: FOLLOWER
I20260811 14:39:34.222292 3398 consensus_queue.cc:260] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a516f0f93d4c72b9e140b57bcfed7f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 37795 } } peers { permanent_uuid: "7451935e26554c93876f66657fe47621" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35629 } } peers { permanent_uuid: "28c1866178d846a58d02d9b5c3b31d4d" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46767 } }
I20260811 14:39:34.222553 3398 ts_tablet_manager.cc:1428] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:39:34.222658 3370 heartbeater.cc:503] Master 127.2.103.126:38583 was elected leader, sending a full tablet report...
W20260811 14:39:34.290889 3143 tablet.cc:2369] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:39:34.398988 2907 tablet.cc:2369] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:39:34.426172 3402 raft_consensus.cc:482] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:39:34.426316 3402 raft_consensus.cc:504] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a516f0f93d4c72b9e140b57bcfed7f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 37795 } } peers { permanent_uuid: "7451935e26554c93876f66657fe47621" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35629 } } peers { permanent_uuid: "28c1866178d846a58d02d9b5c3b31d4d" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46767 } }
I20260811 14:39:34.426877 3402 leader_election.cc:288] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28a516f0f93d4c72b9e140b57bcfed7f (127.2.103.66:37795), 28c1866178d846a58d02d9b5c3b31d4d (127.2.103.67:46767)
I20260811 14:39:34.429934 3388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:34.427188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:51051 (local address 127.2.103.66:37795)
0811 14:39:34.427252 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:34.427253 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:34.427296 (+ 43us) server_negotiation.cc:407] Connection header received
0811 14:39:34.427390 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:34.427391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:34.427405 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:34.427454 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:34.427629 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.428165 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.429187 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.429558 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.429648 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:34.429656 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:34.429811 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 14:39:34.429852 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 14:39:34.429942 3391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:34.427209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.103.65:47829 (local address 127.2.103.67:46767)
0811 14:39:34.427257 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:34.427259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:34.427379 (+ 120us) server_negotiation.cc:407] Connection header received
0811 14:39:34.427461 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:34.427462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:34.427479 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:34.427545 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:39:34.427713 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.428197 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.429180 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.429559 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.429647 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:34.429656 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:34.429821 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 14:39:34.429863 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:39:34.429957 3408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:34.427076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.66:37795 (local address 127.2.103.65:51051)
0811 14:39:34.427229 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:34.427243 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:34.427314 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:34.427471 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:34.427472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:34.427474 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:34.427555 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:34.427557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.428267 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.428268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:34.429078 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:34.429080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.429651 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.429652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:34.429734 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:34.429737 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:39:34.429805 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 14:39:34.429853 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260811 14:39:34.429963 3409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:34.427146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.67:46767 (local address 127.2.103.65:47829)
0811 14:39:34.427292 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:34.427308 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 14:39:34.427396 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:39:34.427551 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:39:34.427552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:39:34.427554 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:39:34.427639 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:34.427641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.428292 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.428293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:34.429083 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:39:34.429085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.429647 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.429648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:39:34.429730 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:34.429734 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:39:34.429811 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 14:39:34.429863 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":17,"thread_start_us":69,"threads_started":1}
I20260811 14:39:34.430327 3061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7025dfb4210492f8aa5bce8c0e9fd73" candidate_uuid: "7451935e26554c93876f66657fe47621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28a516f0f93d4c72b9e140b57bcfed7f" is_pre_election: true
I20260811 14:39:34.430514 3061 raft_consensus.cc:2454] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7451935e26554c93876f66657fe47621 in term 0.
I20260811 14:39:34.430516 3280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7025dfb4210492f8aa5bce8c0e9fd73" candidate_uuid: "7451935e26554c93876f66657fe47621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28c1866178d846a58d02d9b5c3b31d4d" is_pre_election: true
I20260811 14:39:34.430724 3280 raft_consensus.cc:2454] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7451935e26554c93876f66657fe47621 in term 0.
I20260811 14:39:34.430848 2709 leader_election.cc:302] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28a516f0f93d4c72b9e140b57bcfed7f, 7451935e26554c93876f66657fe47621; no voters:
I20260811 14:39:34.430996 3402 raft_consensus.cc:2781] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:39:34.431054 3402 raft_consensus.cc:482] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:39:34.431071 3402 raft_consensus.cc:3037] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:34.431811 3402 raft_consensus.cc:504] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a516f0f93d4c72b9e140b57bcfed7f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 37795 } } peers { permanent_uuid: "7451935e26554c93876f66657fe47621" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35629 } } peers { permanent_uuid: "28c1866178d846a58d02d9b5c3b31d4d" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46767 } }
I20260811 14:39:34.431973 3402 leader_election.cc:288] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [CANDIDATE]: Term 1 election: Requested vote from peers 28a516f0f93d4c72b9e140b57bcfed7f (127.2.103.66:37795), 28c1866178d846a58d02d9b5c3b31d4d (127.2.103.67:46767)
I20260811 14:39:34.432106 3061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7025dfb4210492f8aa5bce8c0e9fd73" candidate_uuid: "7451935e26554c93876f66657fe47621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28a516f0f93d4c72b9e140b57bcfed7f"
I20260811 14:39:34.432140 3280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7025dfb4210492f8aa5bce8c0e9fd73" candidate_uuid: "7451935e26554c93876f66657fe47621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28c1866178d846a58d02d9b5c3b31d4d"
I20260811 14:39:34.432215 3061 raft_consensus.cc:3037] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:34.432219 3280 raft_consensus.cc:3037] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:34.433019 3280 raft_consensus.cc:2454] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7451935e26554c93876f66657fe47621 in term 1.
I20260811 14:39:34.433115 3061 raft_consensus.cc:2454] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7451935e26554c93876f66657fe47621 in term 1.
I20260811 14:39:34.433228 2708 leader_election.cc:302] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28c1866178d846a58d02d9b5c3b31d4d, 7451935e26554c93876f66657fe47621; no voters:
I20260811 14:39:34.433326 3402 raft_consensus.cc:2781] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:34.433501 3402 raft_consensus.cc:686] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [term 1 LEADER]: Becoming Leader. State: Replica: 7451935e26554c93876f66657fe47621, State: Running, Role: LEADER
I20260811 14:39:34.433646 3402 consensus_queue.cc:237] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a516f0f93d4c72b9e140b57bcfed7f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 37795 } } peers { permanent_uuid: "7451935e26554c93876f66657fe47621" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35629 } } peers { permanent_uuid: "28c1866178d846a58d02d9b5c3b31d4d" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46767 } }
I20260811 14:39:34.434566 2572 catalog_manager.cc:5495] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7451935e26554c93876f66657fe47621 (127.2.103.65). New cstate: current_term: 1 leader_uuid: "7451935e26554c93876f66657fe47621" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a516f0f93d4c72b9e140b57bcfed7f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 37795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7451935e26554c93876f66657fe47621" member_type: VOTER last_known_addr { host: "127.2.103.65" port: 35629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28c1866178d846a58d02d9b5c3b31d4d" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46767 } health_report { overall_health: UNKNOWN } } }
W20260811 14:39:34.436976 3372 tablet.cc:2369] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:39:34.441668 3392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:34.438910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50396 (local address 127.2.103.65:35629)
0811 14:39:34.438966 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:34.438969 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:34.439013 (+ 44us) server_negotiation.cc:407] Connection header received
0811 14:39:34.439110 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:34.439110 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:34.439133 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:34.439217 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:39:34.439377 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.439848 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.440757 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:34.440976 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:34.441056 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:34.441223 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:39:34.441415 (+ 192us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:39:34.441537 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:34.441556 (+ 19us) server_negotiation.cc:299] Negotiation successful
0811 14:39:34.441579 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 14:39:34.443660 3061 raft_consensus.cc:1261] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f [term 1 FOLLOWER]: Refusing update from remote peer 7451935e26554c93876f66657fe47621: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:39:34.443707 3280 raft_consensus.cc:1261] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d [term 1 FOLLOWER]: Refusing update from remote peer 7451935e26554c93876f66657fe47621: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:39:34.444060 3402 consensus_queue.cc:1035] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28a516f0f93d4c72b9e140b57bcfed7f" member_type: VOTER last_known_addr { host: "127.2.103.66" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:39:34.444258 3412 consensus_queue.cc:1035] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28c1866178d846a58d02d9b5c3b31d4d" member_type: VOTER last_known_addr { host: "127.2.103.67" port: 46767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:39:34.449222 2571 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:39:34.449360 2571 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:39:34.450181 2571 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb3a94ba03a74ce0ae3888047bde2324: Sending DeleteTablet for 3 replicas of tablet b7025dfb4210492f8aa5bce8c0e9fd73
I20260811 14:39:34.450654 2791 tablet_service.cc:1514] Processing DeleteTablet for tablet b7025dfb4210492f8aa5bce8c0e9fd73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:39:34 UTC) from {username='slave'} at 127.0.0.1:50390
I20260811 14:39:34.450670 3028 tablet_service.cc:1514] Processing DeleteTablet for tablet b7025dfb4210492f8aa5bce8c0e9fd73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:39:34 UTC) from {username='slave'} at 127.0.0.1:58230
I20260811 14:39:34.450903 3430 tablet_replica.cc:330] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f: stopping tablet replica
I20260811 14:39:34.450907 3245 tablet_service.cc:1514] Processing DeleteTablet for tablet b7025dfb4210492f8aa5bce8c0e9fd73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:39:34 UTC) from {username='slave'} at 127.0.0.1:39220
I20260811 14:39:34.450997 3430 raft_consensus.cc:2229] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:39:34.451100 3430 raft_consensus.cc:2258] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:39:34.451133 3431 tablet_replica.cc:330] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d: stopping tablet replica
I20260811 14:39:34.451231 3431 raft_consensus.cc:2229] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:39:34.451474 3429 tablet_replica.cc:330] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621: stopping tablet replica
I20260811 14:39:34.451581 3429 raft_consensus.cc:2229] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:39:34.451601 3430 ts_tablet_manager.cc:1905] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:39:34.451642 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 2683
I20260811 14:39:34.451737 3429 raft_consensus.cc:2258] T b7025dfb4210492f8aa5bce8c0e9fd73 P 7451935e26554c93876f66657fe47621 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:39:34.453563 3430 ts_tablet_manager.cc:1918] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:39:34.453641 3430 log.cc:1196] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-1/wal/wals/b7025dfb4210492f8aa5bce8c0e9fd73
I20260811 14:39:34.453886 3430 ts_tablet_manager.cc:1939] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28a516f0f93d4c72b9e140b57bcfed7f: Deleting consensus metadata
I20260811 14:39:34.454327 2556 catalog_manager.cc:4841] TS 28a516f0f93d4c72b9e140b57bcfed7f (127.2.103.66:37795): tablet b7025dfb4210492f8aa5bce8c0e9fd73 (table test-table [id=2b3dee080ac0432fac1445ebcf5b2e8e]) successfully deleted
I20260811 14:39:34.451329 3431 raft_consensus.cc:2258] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:39:34.465453 3431 ts_tablet_manager.cc:1905] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:39:34.467552 3431 ts_tablet_manager.cc:1918] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:39:34.467654 3431 log.cc:1196] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d: Deleting WAL directory at /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458919184199-2461-0/minicluster-data/ts-2/wal/wals/b7025dfb4210492f8aa5bce8c0e9fd73
I20260811 14:39:34.467929 3431 ts_tablet_manager.cc:1939] T b7025dfb4210492f8aa5bce8c0e9fd73 P 28c1866178d846a58d02d9b5c3b31d4d: Deleting consensus metadata
I20260811 14:39:34.468564 2553 catalog_manager.cc:4841] TS 28c1866178d846a58d02d9b5c3b31d4d (127.2.103.67:46767): tablet b7025dfb4210492f8aa5bce8c0e9fd73 (table test-table [id=2b3dee080ac0432fac1445ebcf5b2e8e]) successfully deleted
W20260811 14:39:34.471585 2556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.103.65:35629 (error 108)
I20260811 14:39:34.471810 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 2910
I20260811 14:39:34.479926 3387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:34.472054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.65:35629 (local address 127.0.0.1:50408)
0811 14:39:34.479763 (+ 7709us) negotiation.cc:107] Waiting for socket to connect
0811 14:39:34.479863 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.65:35629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7663}
W20260811 14:39:34.483170 2556 catalog_manager.cc:4584] TS 7451935e26554c93876f66657fe47621 (127.2.103.65:35629): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b7025dfb4210492f8aa5bce8c0e9fd73: Network error: Client connection negotiation failed: client connection to 127.2.103.65:35629: connect: Connection refused (error 111)
I20260811 14:39:34.489939 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 3146
I20260811 14:39:34.506148 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 2524
2026-08-11T14:39:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (927 ms)
[----------] 6 tests from EncryptionPolicyTest (10057 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:39:34 dist-test-slave-twwt krb5kdc[3439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:39:34 dist-test-slave-twwt krb5kdc[3439](info): set up 2 sockets
Aug 11 14:39:34 dist-test-slave-twwt krb5kdc[3439](info): commencing operation
krb5kdc: starting...
W20260811 14:39:35.827806 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:39:35 dist-test-slave-twwt krb5kdc[3439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:39:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:39:35.966120 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39103
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:36.052191 3460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:36.052451 3460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:36.052495 3460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:36.053685 3460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:36.053759 3460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:36.053784 3460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:36.053807 3460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:36.053836 3460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:36.055408 3460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39103
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:36.055701 3460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:36.055962 3460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:36.061195 3466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:36.061214 3469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:36.061195 3460 server_base.cc:1034] running on GCE node
W20260811 14:39:36.061219 3472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:36.061813 3460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:36.062117 3460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:36.063265 3460 hybrid_clock.cc:648] HybridClock initialized: now 1786459176063241 us; error 46 us; skew 500 ppm
Aug 11 14:39:36 dist-test-slave-twwt krb5kdc[3439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459176, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:36.077584 3460 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:39:36.078114 3460 webserver.cc:460] Webserver started at http://127.0.0.1:33127/ using document root <none> and password file <none>
I20260811 14:39:36.078408 3460 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:36.078485 3460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:36.078701 3460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:36.079898 3460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "cbe1f4c7d23a4013bcfcbc10a811af58"
format_stamp: "Formatted at 2026-08-11 14:39:36 on dist-test-slave-twwt"
server_key: "8bb7cebba0057376d2584276344b993b"
server_key_iv: "dfcca46f981e2ffad63dde30260c8ad7"
server_key_version: "encryptionkey@0"
I20260811 14:39:36.080291 3460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "cbe1f4c7d23a4013bcfcbc10a811af58"
format_stamp: "Formatted at 2026-08-11 14:39:36 on dist-test-slave-twwt"
server_key: "8bb7cebba0057376d2584276344b993b"
server_key_iv: "dfcca46f981e2ffad63dde30260c8ad7"
server_key_version: "encryptionkey@0"
I20260811 14:39:36.081772 3460 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:39:36.082690 3481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:36.082904 3460 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:36.083024 3460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "cbe1f4c7d23a4013bcfcbc10a811af58"
format_stamp: "Formatted at 2026-08-11 14:39:36 on dist-test-slave-twwt"
server_key: "8bb7cebba0057376d2584276344b993b"
server_key_iv: "dfcca46f981e2ffad63dde30260c8ad7"
server_key_version: "encryptionkey@0"
I20260811 14:39:36.083109 3460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:36.111696 3460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:36.114213 3460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:36.114398 3460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:36.118695 3460 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39103
I20260811 14:39:36.118741 3573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39103 every 8 connection(s)
I20260811 14:39:36.119143 3460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:36.120200 3574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:36.122542 3574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58: Bootstrap starting.
I20260811 14:39:36.122757 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 3460
I20260811 14:39:36.122905 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:36.123092 2461 external_mini_cluster.cc:1428] Setting key a19de4918a2f595cf872685c1e61b311
I20260811 14:39:36.123354 3574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:36.123718 3574 log.cc:824] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:36.124547 3574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58: No bootstrap required, opened a new log
I20260811 14:39:36.127125 3574 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe1f4c7d23a4013bcfcbc10a811af58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39103 } }
I20260811 14:39:36.127338 3574 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:36.127374 3574 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe1f4c7d23a4013bcfcbc10a811af58, State: Initialized, Role: FOLLOWER
I20260811 14:39:36.127501 3574 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe1f4c7d23a4013bcfcbc10a811af58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39103 } }
I20260811 14:39:36.127573 3574 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:36.127606 3574 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:36.127653 3574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:36.128366 3574 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe1f4c7d23a4013bcfcbc10a811af58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39103 } }
I20260811 14:39:36.128502 3574 leader_election.cc:302] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbe1f4c7d23a4013bcfcbc10a811af58; no voters:
I20260811 14:39:36.128722 3574 leader_election.cc:288] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:36.128762 3584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:36.128878 3584 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58 [term 1 LEADER]: Becoming Leader. State: Replica: cbe1f4c7d23a4013bcfcbc10a811af58, State: Running, Role: LEADER
I20260811 14:39:36.128961 3584 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe1f4c7d23a4013bcfcbc10a811af58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39103 } }
I20260811 14:39:36.129035 3574 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:36.129304 3584 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbe1f4c7d23a4013bcfcbc10a811af58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe1f4c7d23a4013bcfcbc10a811af58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39103 } } }
I20260811 14:39:36.129387 3584 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:36.129529 3584 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbe1f4c7d23a4013bcfcbc10a811af58. Latest consensus state: current_term: 1 leader_uuid: "cbe1f4c7d23a4013bcfcbc10a811af58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe1f4c7d23a4013bcfcbc10a811af58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39103 } } }
I20260811 14:39:36.129601 3584 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:36.129968 3590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:36.130518 3590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 14:39:36 dist-test-slave-twwt krb5kdc[3439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 14:39:36.132654 3590 catalog_manager.cc:1347] Generated new cluster ID: 05387dee717d4a1ba6bde0a6c089f224
I20260811 14:39:36.132727 3590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:39:36.135500 3582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:36.123725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34309 (local address 127.0.0.1:39103)
0811 14:39:36.123967 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:36.123978 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:36.124017 (+ 39us) server_negotiation.cc:407] Connection header received
0811 14:39:36.124558 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:36.124567 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:36.124621 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:36.124796 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:36.125689 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:36.126669 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:36.127719 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:36.128164 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:36.128249 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:36.131541 (+ 3292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:36.131546 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:36.131547 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:36.131560 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:36.133645 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:36.134125 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:36.134129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:36.134129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:36.134248 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:36.134492 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:36.134493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:36.134493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:36.134866 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:36.135057 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:36.135154 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 14:39:36.135263 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":104,"threads_started":1}
I20260811 14:39:36.136658 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:39:36.145433 3582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:36.138134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43853 (local address 127.0.0.1:39103)
0811 14:39:36.138177 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:36.138179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:36.138274 (+ 95us) server_negotiation.cc:407] Connection header received
0811 14:39:36.138322 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:36.138323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:36.138336 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:36.138391 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:36.139031 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:36.139812 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:36.140796 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:36.141163 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:36.141239 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:36.142689 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:36.142690 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:36.142690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:36.142697 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:36.144249 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:36.144640 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:36.144641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:36.144642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:36.144689 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:36.144898 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:36.144899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:36.144899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:36.145012 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:36.145082 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:36.145279 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 14:39:36.145341 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:39:36.168362 3590 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:36.169001 3590 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:36.182578 3590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbe1f4c7d23a4013bcfcbc10a811af58: Generated new TSK 0
I20260811 14:39:36.182889 3590 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:39:36.192086 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 3460
W20260811 14:39:36.208232 3433 connection.cc:535] client connection to 127.0.0.1:39103 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39103): Connection reset by peer (error 104)
2026-08-11T14:39:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1690 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:39:36 dist-test-slave-twwt krb5kdc[3625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:39:36 dist-test-slave-twwt krb5kdc[3625](info): set up 2 sockets
Aug 11 14:39:36 dist-test-slave-twwt krb5kdc[3625](info): commencing operation
krb5kdc: starting...
W20260811 14:39:37.468429 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:39:37 dist-test-slave-twwt krb5kdc[3625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:39:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:39:37.601717 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44237
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:37.687744 3645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:37.687973 3645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:37.688000 3645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:37.689091 3645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:37.689126 3645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:37.689141 3645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:37.689153 3645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:37.689165 3645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:37.690586 3645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44237
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:37.690908 3645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:37.691160 3645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:37.696153 3654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:37.696286 3656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:37.696131 3653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:37.698154 3645 server_base.cc:1034] running on GCE node
I20260811 14:39:37.698360 3645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:37.698683 3645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:37.699836 3645 hybrid_clock.cc:648] HybridClock initialized: now 1786459177699820 us; error 31 us; skew 500 ppm
Aug 11 14:39:37 dist-test-slave-twwt krb5kdc[3625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459177, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:37.702319 3645 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:39:37.702848 3645 webserver.cc:460] Webserver started at http://127.0.0.1:41227/ using document root <none> and password file <none>
I20260811 14:39:37.703110 3645 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:37.703157 3645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:37.703316 3645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:37.704514 3645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "1fae60243a1c4d259cfe3da313c75aa8"
format_stamp: "Formatted at 2026-08-11 14:39:37 on dist-test-slave-twwt"
server_key: "012e97c42604a6b593123aedb59edef0"
server_key_iv: "d501c8f92136462f71a377c05043c3c2"
server_key_version: "encryptionkey@0"
I20260811 14:39:37.704885 3645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "1fae60243a1c4d259cfe3da313c75aa8"
format_stamp: "Formatted at 2026-08-11 14:39:37 on dist-test-slave-twwt"
server_key: "012e97c42604a6b593123aedb59edef0"
server_key_iv: "d501c8f92136462f71a377c05043c3c2"
server_key_version: "encryptionkey@0"
I20260811 14:39:37.706334 3645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 14:39:37.707367 3674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:37.707639 3645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:39:37.707770 3645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "1fae60243a1c4d259cfe3da313c75aa8"
format_stamp: "Formatted at 2026-08-11 14:39:37 on dist-test-slave-twwt"
server_key: "012e97c42604a6b593123aedb59edef0"
server_key_iv: "d501c8f92136462f71a377c05043c3c2"
server_key_version: "encryptionkey@0"
I20260811 14:39:37.707861 3645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:37.723877 3645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:37.725900 3645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:37.726084 3645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:37.730738 3645 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44237
I20260811 14:39:37.730783 3755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44237 every 8 connection(s)
I20260811 14:39:37.731153 3645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:37.732152 3757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:37.734551 3757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8: Bootstrap starting.
I20260811 14:39:37.735180 3757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:37.735445 3757 log.cc:824] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:37.736224 3757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8: No bootstrap required, opened a new log
I20260811 14:39:37.737843 3757 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fae60243a1c4d259cfe3da313c75aa8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44237 } }
I20260811 14:39:37.737989 3757 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:37.738013 3757 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fae60243a1c4d259cfe3da313c75aa8, State: Initialized, Role: FOLLOWER
I20260811 14:39:37.738194 3757 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fae60243a1c4d259cfe3da313c75aa8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44237 } }
I20260811 14:39:37.738270 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 3645
I20260811 14:39:37.738319 3757 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:37.738360 3757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:37.738384 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:37.738416 3757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:37.738597 2461 external_mini_cluster.cc:1428] Setting key 2b04bdee0c2e8c9fb93810c79fb4f4da
I20260811 14:39:37.739128 3757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fae60243a1c4d259cfe3da313c75aa8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44237 } }
I20260811 14:39:37.739339 3757 leader_election.cc:302] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fae60243a1c4d259cfe3da313c75aa8; no voters:
I20260811 14:39:37.739534 3757 leader_election.cc:288] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:37.739562 3763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:37.739674 3763 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8 [term 1 LEADER]: Becoming Leader. State: Replica: 1fae60243a1c4d259cfe3da313c75aa8, State: Running, Role: LEADER
I20260811 14:39:37.739820 3757 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:37.739835 3763 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fae60243a1c4d259cfe3da313c75aa8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44237 } }
I20260811 14:39:37.740563 3767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fae60243a1c4d259cfe3da313c75aa8. Latest consensus state: current_term: 1 leader_uuid: "1fae60243a1c4d259cfe3da313c75aa8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fae60243a1c4d259cfe3da313c75aa8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44237 } } }
I20260811 14:39:37.740674 3767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:37.740911 3766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fae60243a1c4d259cfe3da313c75aa8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fae60243a1c4d259cfe3da313c75aa8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44237 } } }
I20260811 14:39:37.741066 3766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:37.741345 3781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:37.742086 3781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:37.744414 3781 catalog_manager.cc:1347] Generated new cluster ID: 0aadf2e0842f44b69b43578ade4d489f
I20260811 14:39:37.744479 3781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:39:37.745759 3761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:37.739152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48251 (local address 127.0.0.1:44237)
0811 14:39:37.739369 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:37.739381 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:37.739422 (+ 41us) server_negotiation.cc:407] Connection header received
0811 14:39:37.739560 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:37.739567 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:37.739653 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:37.739833 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:37.740232 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:37.742216 (+ 1984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:37.743476 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:37.744106 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:37.744225 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:37.745021 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:37.745025 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:37.745025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:37.745208 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:37.745249 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:37.745260 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:37.745371 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:37.745458 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 14:39:37.745589 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":149,"thread_start_us":101,"threads_started":1}
I20260811 14:39:37.746788 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:39:37.753854 3781 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:37.754638 3781 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:37.754648 3761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:37.751022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57351 (local address 127.0.0.1:44237)
0811 14:39:37.751068 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:37.751070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:37.751360 (+ 290us) server_negotiation.cc:407] Connection header received
0811 14:39:37.751450 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:37.751451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:37.751466 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:37.751540 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:37.751725 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:37.752277 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:37.753107 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:37.753420 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:37.753701 (+ 281us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:37.754299 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:37.754301 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:37.754301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:37.754306 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:37.754324 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:37.754328 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:37.754413 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:37.754491 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 14:39:37.754504 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 14:39:37.759105 3781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fae60243a1c4d259cfe3da313c75aa8: Generated new TSK 0
I20260811 14:39:37.759361 3781 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:39:37.761667 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 3645
W20260811 14:39:37.775638 3622 connection.cc:535] client connection to 127.0.0.1:44237 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44237): Connection reset by peer (error 104)
2026-08-11T14:39:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1567 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:39:37 dist-test-slave-twwt krb5kdc[3806](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:39:37 dist-test-slave-twwt krb5kdc[3806](info): set up 2 sockets
Aug 11 14:39:37 dist-test-slave-twwt krb5kdc[3806](info): commencing operation
krb5kdc: starting...
W20260811 14:39:39.090971 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:39:39 dist-test-slave-twwt krb5kdc[3806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:39:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:39:39.229436 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43123
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:39.329571 3828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:39.329782 3828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:39.329804 3828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:39.330888 3828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:39.330924 3828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:39.330937 3828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:39.330950 3828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:39.330962 3828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:39.332424 3828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43123
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:39.332623 3828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:39.332813 3828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:39.337880 3845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:39.337857 3846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:39.337891 3848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:39.338215 3828 server_base.cc:1034] running on GCE node
I20260811 14:39:39.338498 3828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:39.338922 3828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:39.340071 3828 hybrid_clock.cc:648] HybridClock initialized: now 1786459179340051 us; error 42 us; skew 500 ppm
Aug 11 14:39:39 dist-test-slave-twwt krb5kdc[3806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459179, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:39.342852 3828 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:39:39.343417 3828 webserver.cc:460] Webserver started at http://127.0.0.1:37507/ using document root <none> and password file <none>
I20260811 14:39:39.343667 3828 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:39.343715 3828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:39.343858 3828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:39.345016 3828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "cfb3711af3b04919aa6b62bfaf69cb8f"
format_stamp: "Formatted at 2026-08-11 14:39:39 on dist-test-slave-twwt"
server_key: "74b9188e14f6898040a0cce70bdafa91"
server_key_iv: "854b283356b77be6064242ac3466d9ec"
server_key_version: "encryptionkey@0"
I20260811 14:39:39.345405 3828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "cfb3711af3b04919aa6b62bfaf69cb8f"
format_stamp: "Formatted at 2026-08-11 14:39:39 on dist-test-slave-twwt"
server_key: "74b9188e14f6898040a0cce70bdafa91"
server_key_iv: "854b283356b77be6064242ac3466d9ec"
server_key_version: "encryptionkey@0"
I20260811 14:39:39.346967 3828 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:39:39.347823 3857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:39.348006 3828 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:39.348124 3828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "cfb3711af3b04919aa6b62bfaf69cb8f"
format_stamp: "Formatted at 2026-08-11 14:39:39 on dist-test-slave-twwt"
server_key: "74b9188e14f6898040a0cce70bdafa91"
server_key_iv: "854b283356b77be6064242ac3466d9ec"
server_key_version: "encryptionkey@0"
I20260811 14:39:39.348215 3828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:39.351822 3828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:39.353864 3828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:39.354031 3828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:39.358471 3828 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43123
I20260811 14:39:39.358506 3948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43123 every 8 connection(s)
I20260811 14:39:39.358943 3828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:39.359972 3949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:39.362295 3949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f: Bootstrap starting.
I20260811 14:39:39.362959 3949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:39.363204 3949 log.cc:824] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:39.363909 3949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f: No bootstrap required, opened a new log
I20260811 14:39:39.365494 3949 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb3711af3b04919aa6b62bfaf69cb8f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43123 } }
I20260811 14:39:39.365639 3949 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:39.365662 3949 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfb3711af3b04919aa6b62bfaf69cb8f, State: Initialized, Role: FOLLOWER
I20260811 14:39:39.365810 3949 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb3711af3b04919aa6b62bfaf69cb8f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43123 } }
I20260811 14:39:39.365916 3949 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:39.365963 3949 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:39.366039 3949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:39.366079 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 3828
I20260811 14:39:39.366215 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:39.366430 2461 external_mini_cluster.cc:1428] Setting key 5e9332a43edca3aa6a8ae6cd21f0d0bb
I20260811 14:39:39.366838 3949 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb3711af3b04919aa6b62bfaf69cb8f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43123 } }
I20260811 14:39:39.366952 3949 leader_election.cc:302] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfb3711af3b04919aa6b62bfaf69cb8f; no voters:
I20260811 14:39:39.367121 3949 leader_election.cc:288] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:39.367187 3957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:39.367329 3957 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f [term 1 LEADER]: Becoming Leader. State: Replica: cfb3711af3b04919aa6b62bfaf69cb8f, State: Running, Role: LEADER
I20260811 14:39:39.367429 3949 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:39.367470 3957 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb3711af3b04919aa6b62bfaf69cb8f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43123 } }
I20260811 14:39:39.367803 3957 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfb3711af3b04919aa6b62bfaf69cb8f. Latest consensus state: current_term: 1 leader_uuid: "cfb3711af3b04919aa6b62bfaf69cb8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb3711af3b04919aa6b62bfaf69cb8f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43123 } } }
I20260811 14:39:39.367883 3957 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:39.368031 3961 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfb3711af3b04919aa6b62bfaf69cb8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb3711af3b04919aa6b62bfaf69cb8f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43123 } } }
I20260811 14:39:39.368119 3961 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:39.368175 3968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:39.369140 3968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:39.370079 3956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:39.366964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35307 (local address 127.0.0.1:43123)
0811 14:39:39.367168 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:39.367178 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:39.367221 (+ 43us) server_negotiation.cc:407] Connection header received
0811 14:39:39.367352 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:39.367359 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:39.367389 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:39.367567 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:39.369440 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:39.369444 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:39.369444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:39.369459 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:39.369500 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:39.369514 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:39.369586 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:39.369844 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 14:39:39.369890 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":108,"threads_started":1}
I20260811 14:39:39.371194 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:39:39.371407 3968 catalog_manager.cc:1347] Generated new cluster ID: a7582efe74a8496f92fb333df434cfd9
I20260811 14:39:39.371486 3968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:39:39.383944 3956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:39.372748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54495 (local address 127.0.0.1:43123)
0811 14:39:39.372808 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:39.372810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:39.382836 (+ 10026us) server_negotiation.cc:407] Connection header received
0811 14:39:39.382921 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:39.382922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:39.382930 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:39.382985 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:39.383690 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:39.383692 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:39.383692 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:39.383697 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:39.383714 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:39.383719 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:39.383776 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:39.383838 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 14:39:39.383852 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 14:39:39.387100 3968 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:39.387300 3968 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:39.391601 3968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfb3711af3b04919aa6b62bfaf69cb8f: Generated new TSK 0
I20260811 14:39:39.391808 3968 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:39:39.394124 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 3828
2026-08-11T14:39:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1630 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:39:39 dist-test-slave-twwt krb5kdc[4003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:39:39 dist-test-slave-twwt krb5kdc[4003](info): set up 2 sockets
Aug 11 14:39:39 dist-test-slave-twwt krb5kdc[4003](info): commencing operation
krb5kdc: starting...
W20260811 14:39:40.670838 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:39:40 dist-test-slave-twwt krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:39:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:39:40.810416 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34111
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:40.897150 4025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:40.897406 4025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:40.897447 4025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:40.898639 4025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:40.898713 4025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:40.898737 4025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:40.898757 4025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:40.898785 4025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:40.900286 4025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:40.900580 4025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:40.900823 4025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:40.906297 4035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:40.906352 4031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:40.906533 4025 server_base.cc:1034] running on GCE node
W20260811 14:39:40.906312 4032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:40.906975 4025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:40.907364 4025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:40.908552 4025 hybrid_clock.cc:648] HybridClock initialized: now 1786459180908529 us; error 44 us; skew 500 ppm
Aug 11 14:39:40 dist-test-slave-twwt krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459180, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:40.911605 4025 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:39:40.912107 4025 webserver.cc:460] Webserver started at http://127.0.0.1:38627/ using document root <none> and password file <none>
I20260811 14:39:40.912352 4025 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:40.912400 4025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:40.912498 4025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:40.913666 4025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "270cbb915ac844f2b8694aca6382462c"
format_stamp: "Formatted at 2026-08-11 14:39:40 on dist-test-slave-twwt"
server_key: "b6d14fb7cdeb8615ad3542e35c8c8c2e"
server_key_iv: "7f82c6ba130f4688c9e0d5603f0febf0"
server_key_version: "encryptionkey@0"
I20260811 14:39:40.914004 4025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "270cbb915ac844f2b8694aca6382462c"
format_stamp: "Formatted at 2026-08-11 14:39:40 on dist-test-slave-twwt"
server_key: "b6d14fb7cdeb8615ad3542e35c8c8c2e"
server_key_iv: "7f82c6ba130f4688c9e0d5603f0febf0"
server_key_version: "encryptionkey@0"
I20260811 14:39:40.915624 4025 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 14:39:40.916600 4051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:40.916832 4025 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:40.916931 4025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "270cbb915ac844f2b8694aca6382462c"
format_stamp: "Formatted at 2026-08-11 14:39:40 on dist-test-slave-twwt"
server_key: "b6d14fb7cdeb8615ad3542e35c8c8c2e"
server_key_iv: "7f82c6ba130f4688c9e0d5603f0febf0"
server_key_version: "encryptionkey@0"
I20260811 14:39:40.916996 4025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:40.943090 4025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:40.945586 4025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:40.945717 4025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:40.950227 4025 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34111
I20260811 14:39:40.950260 4134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34111 every 8 connection(s)
I20260811 14:39:40.950691 4025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:40.951881 4135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:40.954538 4135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c: Bootstrap starting.
I20260811 14:39:40.955255 4135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:40.955587 4135 log.cc:824] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:40.956423 4135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c: No bootstrap required, opened a new log
I20260811 14:39:40.957315 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 4025
I20260811 14:39:40.957448 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:40.957643 2461 external_mini_cluster.cc:1428] Setting key 9cfb659de7c1ac3f871f68c976a6a604
I20260811 14:39:40.958845 4135 raft_consensus.cc:348] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270cbb915ac844f2b8694aca6382462c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34111 } }
I20260811 14:39:40.959039 4135 raft_consensus.cc:374] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:40.959097 4135 raft_consensus.cc:729] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 270cbb915ac844f2b8694aca6382462c, State: Initialized, Role: FOLLOWER
I20260811 14:39:40.959251 4135 consensus_queue.cc:260] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270cbb915ac844f2b8694aca6382462c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34111 } }
I20260811 14:39:40.959345 4135 raft_consensus.cc:388] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:40.959383 4135 raft_consensus.cc:482] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:40.959441 4135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:40.960506 4135 raft_consensus.cc:504] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270cbb915ac844f2b8694aca6382462c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34111 } }
I20260811 14:39:40.960608 4135 leader_election.cc:302] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 270cbb915ac844f2b8694aca6382462c; no voters:
I20260811 14:39:40.960846 4135 leader_election.cc:288] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:40.960871 4143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:40.960988 4143 raft_consensus.cc:686] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c [term 1 LEADER]: Becoming Leader. State: Replica: 270cbb915ac844f2b8694aca6382462c, State: Running, Role: LEADER
I20260811 14:39:40.961138 4143 consensus_queue.cc:237] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270cbb915ac844f2b8694aca6382462c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34111 } }
I20260811 14:39:40.961143 4135 sys_catalog.cc:564] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:40.961596 4144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 270cbb915ac844f2b8694aca6382462c. Latest consensus state: current_term: 1 leader_uuid: "270cbb915ac844f2b8694aca6382462c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270cbb915ac844f2b8694aca6382462c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34111 } } }
I20260811 14:39:40.961680 4144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:40.961848 4145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "270cbb915ac844f2b8694aca6382462c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270cbb915ac844f2b8694aca6382462c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34111 } } }
I20260811 14:39:40.961982 4145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:40.962461 4147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:40.963105 4147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:40.964847 4147 catalog_manager.cc:1347] Generated new cluster ID: f8c016a91de54e289cdabd8bdb6cc07e
I20260811 14:39:40.964905 4147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:39:40 dist-test-slave-twwt krb5kdc[4003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 14:39:40.969074 4141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:40.958232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57411 (local address 127.0.0.1:34111)
0811 14:39:40.958442 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:40.958450 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:40.958477 (+ 27us) server_negotiation.cc:407] Connection header received
0811 14:39:40.958792 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:40.958799 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:40.958853 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:40.958968 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:40.959860 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:40.960909 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:40.962007 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:40.962604 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:40.962771 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:40.965567 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:40.965571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:40.965571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:40.965585 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:40.967331 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:40.967725 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:40.967729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:40.967730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:40.967856 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:40.968089 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:40.968090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:40.968090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:40.968351 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:40.968518 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:40.968645 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 14:39:40.968906 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":114,"threads_started":1}
I20260811 14:39:40.970342 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:39:40.978739 4147 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:40.979492 4147 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:40.981648 4141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:40.971906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45171 (local address 127.0.0.1:34111)
0811 14:39:40.971969 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:40.971971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:40.974857 (+ 2886us) server_negotiation.cc:407] Connection header received
0811 14:39:40.974932 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:40.974933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:40.974944 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:40.975003 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:40.975568 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:40.976068 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:40.976799 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:40.977030 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:40.977114 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:40.978766 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:40.978767 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:40.978767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:40.978775 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:40.980401 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:40.980793 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:40.980794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:40.980794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:40.980871 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:40.981086 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:40.981087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:40.981088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:40.981202 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:40.981337 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:40.981444 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 14:39:40.981541 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 14:39:40.997350 4147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 270cbb915ac844f2b8694aca6382462c: Generated new TSK 0
I20260811 14:39:40.997649 4147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:39:41.001361 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 4025
2026-08-11T14:39:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1616 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:39:41 dist-test-slave-twwt krb5kdc[4183](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:39:41 dist-test-slave-twwt krb5kdc[4183](info): set up 2 sockets
Aug 11 14:39:41 dist-test-slave-twwt krb5kdc[4183](info): commencing operation
krb5kdc: starting...
W20260811 14:39:42.288452 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:39:42 dist-test-slave-twwt krb5kdc[4183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:39:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:39:42.426079 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35735
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:42.511763 4202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:42.512029 4202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:42.512068 4202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:42.513238 4202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:42.513311 4202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:42.513335 4202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:42.513356 4202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:42.513388 4202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:42.514958 4202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:42.515244 4202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:42.515492 4202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:42.520744 4215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:42.520876 4202 server_base.cc:1034] running on GCE node
W20260811 14:39:42.520756 4209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:42.520766 4210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:42.521224 4202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:42.521559 4202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:42.522727 4202 hybrid_clock.cc:648] HybridClock initialized: now 1786459182522706 us; error 40 us; skew 500 ppm
Aug 11 14:39:42 dist-test-slave-twwt krb5kdc[4183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459182, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:42.529667 4202 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:39:42.530220 4202 webserver.cc:460] Webserver started at http://127.0.0.1:32981/ using document root <none> and password file <none>
I20260811 14:39:42.530480 4202 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:42.530529 4202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:42.530689 4202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:42.531901 4202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "89862206eab745a1970528d0df904ffa"
format_stamp: "Formatted at 2026-08-11 14:39:42 on dist-test-slave-twwt"
server_key: "bea680d31fb07b88749a94508c6db41f"
server_key_iv: "3073fc8b34cbfccb7c8c6311e21581ae"
server_key_version: "encryptionkey@0"
I20260811 14:39:42.532277 4202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "89862206eab745a1970528d0df904ffa"
format_stamp: "Formatted at 2026-08-11 14:39:42 on dist-test-slave-twwt"
server_key: "bea680d31fb07b88749a94508c6db41f"
server_key_iv: "3073fc8b34cbfccb7c8c6311e21581ae"
server_key_version: "encryptionkey@0"
I20260811 14:39:42.534107 4202 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 14:39:42.535023 4227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:42.535202 4202 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:39:42.535303 4202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "89862206eab745a1970528d0df904ffa"
format_stamp: "Formatted at 2026-08-11 14:39:42 on dist-test-slave-twwt"
server_key: "bea680d31fb07b88749a94508c6db41f"
server_key_iv: "3073fc8b34cbfccb7c8c6311e21581ae"
server_key_version: "encryptionkey@0"
I20260811 14:39:42.535375 4202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:42.548702 4202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:42.551061 4202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:42.551222 4202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:42.555397 4202 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35735
I20260811 14:39:42.555469 4309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35735 every 8 connection(s)
I20260811 14:39:42.555833 4202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:42.556902 4310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:42.559381 4310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa: Bootstrap starting.
I20260811 14:39:42.559973 4310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:42.560216 4310 log.cc:824] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:42.560950 4310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa: No bootstrap required, opened a new log
I20260811 14:39:42.562369 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 4202
I20260811 14:39:42.562522 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:42.562767 2461 external_mini_cluster.cc:1428] Setting key 948caaf9359a51a25eb0be7aa6479e35
I20260811 14:39:42.562747 4310 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89862206eab745a1970528d0df904ffa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35735 } }
I20260811 14:39:42.562963 4310 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:42.562992 4310 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89862206eab745a1970528d0df904ffa, State: Initialized, Role: FOLLOWER
I20260811 14:39:42.563108 4310 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89862206eab745a1970528d0df904ffa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35735 } }
I20260811 14:39:42.563184 4310 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:42.563215 4310 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:42.563265 4310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:42.563983 4310 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89862206eab745a1970528d0df904ffa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35735 } }
I20260811 14:39:42.564122 4310 leader_election.cc:302] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89862206eab745a1970528d0df904ffa; no voters:
I20260811 14:39:42.564328 4310 leader_election.cc:288] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:42.564376 4316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:42.564586 4316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa [term 1 LEADER]: Becoming Leader. State: Replica: 89862206eab745a1970528d0df904ffa, State: Running, Role: LEADER
I20260811 14:39:42.564610 4310 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:42.564728 4316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89862206eab745a1970528d0df904ffa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35735 } }
I20260811 14:39:42.565171 4319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89862206eab745a1970528d0df904ffa. Latest consensus state: current_term: 1 leader_uuid: "89862206eab745a1970528d0df904ffa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89862206eab745a1970528d0df904ffa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35735 } } }
I20260811 14:39:42.565177 4318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89862206eab745a1970528d0df904ffa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89862206eab745a1970528d0df904ffa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35735 } } }
I20260811 14:39:42.565318 4319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:42.565323 4318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:42.565572 4330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:42.566102 4330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:42.568109 4330 catalog_manager.cc:1347] Generated new cluster ID: 8ce3edc31593482ca233a21e84bb566b
I20260811 14:39:42.568164 4330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:39:42.569633 4315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:42.563361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51703 (local address 127.0.0.1:35735)
0811 14:39:42.563584 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:42.563593 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:42.563621 (+ 28us) server_negotiation.cc:407] Connection header received
0811 14:39:42.563997 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:42.564003 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:42.564054 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:42.564246 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:42.564507 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:42.566150 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:42.567280 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:42.567889 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:42.568005 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:42.568809 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:42.568813 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:42.568813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:42.569025 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:42.569063 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:42.569077 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:42.569139 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:42.569254 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 14:39:42.569415 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":103,"threads_started":1}
I20260811 14:39:42.570632 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:39:42.575769 4315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:42.572167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48791 (local address 127.0.0.1:35735)
0811 14:39:42.572203 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:42.572205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:42.572344 (+ 139us) server_negotiation.cc:407] Connection header received
0811 14:39:42.572411 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:42.572412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:42.572423 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:42.572476 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:42.572616 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:42.573349 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:42.574450 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:42.574772 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:42.574829 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:42.575538 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:42.575539 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:42.575540 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:42.575545 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:42.575557 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:42.575561 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:42.575602 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:42.575671 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 14:39:42.575695 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 14:39:42.580482 4330 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:42.581019 4330 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:42.583534 4330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89862206eab745a1970528d0df904ffa: Generated new TSK 0
I20260811 14:39:42.583720 4330 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:39:42.587381 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 4202
2026-08-11T14:39:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1584 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:39:42 dist-test-slave-twwt krb5kdc[4355](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:39:42 dist-test-slave-twwt krb5kdc[4355](info): set up 2 sockets
Aug 11 14:39:42 dist-test-slave-twwt krb5kdc[4355](info): commencing operation
krb5kdc: starting...
W20260811 14:39:43.870657 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:39:43 dist-test-slave-twwt krb5kdc[4355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:39:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:39:44.018652 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34791
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:44.104974 4376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:44.105191 4376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:44.105214 4376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:44.106388 4376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:44.106437 4376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:44.106451 4376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:44.106464 4376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:44.106477 4376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:44.108050 4376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:44.108327 4376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:44.108585 4376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:44.113833 4383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:44.113880 4385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:44.113880 4387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:44.113961 4376 server_base.cc:1034] running on GCE node
I20260811 14:39:44.114316 4376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:44.114707 4376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:44.115899 4376 hybrid_clock.cc:648] HybridClock initialized: now 1786459184115876 us; error 44 us; skew 500 ppm
Aug 11 14:39:44 dist-test-slave-twwt krb5kdc[4355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459184, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:44.118789 4376 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:39:44.119210 4376 webserver.cc:460] Webserver started at http://127.0.0.1:38861/ using document root <none> and password file <none>
I20260811 14:39:44.119519 4376 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:44.119573 4376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:44.119675 4376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:44.120824 4376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "aa49f2658039424aa2cb416b5d6c9fb4"
format_stamp: "Formatted at 2026-08-11 14:39:44 on dist-test-slave-twwt"
server_key: "86edaef0b3925c7fb716512f63f26540"
server_key_iv: "2a9a6432cbeba2dda0f5c50497a83e92"
server_key_version: "encryptionkey@0"
I20260811 14:39:44.121145 4376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "aa49f2658039424aa2cb416b5d6c9fb4"
format_stamp: "Formatted at 2026-08-11 14:39:44 on dist-test-slave-twwt"
server_key: "86edaef0b3925c7fb716512f63f26540"
server_key_iv: "2a9a6432cbeba2dda0f5c50497a83e92"
server_key_version: "encryptionkey@0"
I20260811 14:39:44.122550 4376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 14:39:44.123569 4399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:44.123754 4376 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:39:44.123860 4376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "aa49f2658039424aa2cb416b5d6c9fb4"
format_stamp: "Formatted at 2026-08-11 14:39:44 on dist-test-slave-twwt"
server_key: "86edaef0b3925c7fb716512f63f26540"
server_key_iv: "2a9a6432cbeba2dda0f5c50497a83e92"
server_key_version: "encryptionkey@0"
I20260811 14:39:44.123929 4376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:44.127497 4376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:44.129963 4376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:44.130084 4376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:44.134505 4376 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34791
I20260811 14:39:44.134593 4495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34791 every 8 connection(s)
I20260811 14:39:44.135005 4376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:44.136154 4496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:44.138746 4496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4: Bootstrap starting.
I20260811 14:39:44.139387 4496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:44.139696 4496 log.cc:824] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:44.140563 4496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4: No bootstrap required, opened a new log
I20260811 14:39:44.142239 4496 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49f2658039424aa2cb416b5d6c9fb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34791 } }
I20260811 14:39:44.142426 4496 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:44.142478 4496 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa49f2658039424aa2cb416b5d6c9fb4, State: Initialized, Role: FOLLOWER
I20260811 14:39:44.142697 4496 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49f2658039424aa2cb416b5d6c9fb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34791 } }
I20260811 14:39:44.142810 4496 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:44.142856 4496 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:44.142911 4496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:44.143499 4496 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49f2658039424aa2cb416b5d6c9fb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34791 } }
I20260811 14:39:44.143626 4496 leader_election.cc:302] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa49f2658039424aa2cb416b5d6c9fb4; no voters:
I20260811 14:39:44.143828 4496 leader_election.cc:288] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:44.143884 4502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:44.144115 4502 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4 [term 1 LEADER]: Becoming Leader. State: Replica: aa49f2658039424aa2cb416b5d6c9fb4, State: Running, Role: LEADER
I20260811 14:39:44.144124 4496 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:44.144246 4502 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49f2658039424aa2cb416b5d6c9fb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34791 } }
I20260811 14:39:44.144954 4504 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa49f2658039424aa2cb416b5d6c9fb4. Latest consensus state: current_term: 1 leader_uuid: "aa49f2658039424aa2cb416b5d6c9fb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49f2658039424aa2cb416b5d6c9fb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34791 } } }
I20260811 14:39:44.144982 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 4376
I20260811 14:39:44.145046 4504 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:44.145116 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:44.145102 4503 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa49f2658039424aa2cb416b5d6c9fb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa49f2658039424aa2cb416b5d6c9fb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34791 } } }
I20260811 14:39:44.145195 4503 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:44.145340 2461 external_mini_cluster.cc:1428] Setting key acc784da99b876559d3c7b0549d84f6a
I20260811 14:39:44.145354 4515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:44.145903 4515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:44.147876 4515 catalog_manager.cc:1347] Generated new cluster ID: 26dc54a8829a47a0a0e942100e917e7f
I20260811 14:39:44.147933 4515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:39:44.148330 4534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:44.146073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35049 (local address 127.0.0.1:34791)
0811 14:39:44.146257 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:44.146268 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:44.146302 (+ 34us) server_negotiation.cc:407] Connection header received
0811 14:39:44.146577 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:44.146586 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:44.146667 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:44.146825 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:44.147703 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:44.147707 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:44.147707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:44.147711 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:44.147756 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:44.147771 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:44.148079 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:44.148111 (+ 32us) server_negotiation.cc:299] Negotiation successful
0811 14:39:44.148155 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
I20260811 14:39:44.149382 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:39:44.153491 4515 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:44.153684 4515 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:44.155189 4534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:44.153855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37725 (local address 127.0.0.1:34791)
0811 14:39:44.153950 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:44.153952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:44.154085 (+ 133us) server_negotiation.cc:407] Connection header received
0811 14:39:44.154174 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:44.154174 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:44.154180 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:44.154256 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:44.154944 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:44.154945 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:44.154945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:44.154950 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:44.154965 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:44.154970 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:44.155024 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:44.155087 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 14:39:44.155102 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 14:39:44.165087 4515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa49f2658039424aa2cb416b5d6c9fb4: Generated new TSK 0
I20260811 14:39:44.165349 4515 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:39:44.169384 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 4376
2026-08-11T14:39:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1572 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:39:44 dist-test-slave-twwt krb5kdc[4549](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:39:44 dist-test-slave-twwt krb5kdc[4549](info): set up 2 sockets
Aug 11 14:39:44 dist-test-slave-twwt krb5kdc[4549](info): commencing operation
krb5kdc: starting...
W20260811 14:39:45.467311 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:39:45 dist-test-slave-twwt krb5kdc[4549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:39:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:39:45.605659 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45373
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:45.693131 4572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:45.693394 4572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:45.693423 4572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:45.694649 4572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:45.694743 4572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:45.694772 4572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:45.694795 4572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:45.694823 4572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:45.696529 4572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:45.696864 4572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:45.697139 4572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:45.702790 4580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:45.702786 4581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:45.702915 4572 server_base.cc:1034] running on GCE node
W20260811 14:39:45.702816 4583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:45.703311 4572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:45.703679 4572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:45.704910 4572 hybrid_clock.cc:648] HybridClock initialized: now 1786459185704882 us; error 48 us; skew 500 ppm
Aug 11 14:39:45 dist-test-slave-twwt krb5kdc[4549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459185, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:45.707677 4572 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:39:45.708204 4572 webserver.cc:460] Webserver started at http://127.0.0.1:40475/ using document root <none> and password file <none>
I20260811 14:39:45.708462 4572 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:45.708536 4572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:45.708694 4572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:45.709793 4572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "e25bb9c331f14f5bb2cab0569bd25d31"
format_stamp: "Formatted at 2026-08-11 14:39:45 on dist-test-slave-twwt"
server_key: "c06b360583a280f7fcb879d1d4d5a24e"
server_key_iv: "d84b5ac9f10c42b4a65b0fac92d13263"
server_key_version: "encryptionkey@0"
I20260811 14:39:45.710191 4572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "e25bb9c331f14f5bb2cab0569bd25d31"
format_stamp: "Formatted at 2026-08-11 14:39:45 on dist-test-slave-twwt"
server_key: "c06b360583a280f7fcb879d1d4d5a24e"
server_key_iv: "d84b5ac9f10c42b4a65b0fac92d13263"
server_key_version: "encryptionkey@0"
I20260811 14:39:45.711699 4572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:39:45.712594 4595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:45.712956 4572 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:45.713093 4572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "e25bb9c331f14f5bb2cab0569bd25d31"
format_stamp: "Formatted at 2026-08-11 14:39:45 on dist-test-slave-twwt"
server_key: "c06b360583a280f7fcb879d1d4d5a24e"
server_key_iv: "d84b5ac9f10c42b4a65b0fac92d13263"
server_key_version: "encryptionkey@0"
I20260811 14:39:45.713186 4572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:45.731956 4572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:45.734318 4572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:45.734511 4572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:45.739166 4572 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45373
I20260811 14:39:45.739233 4685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45373 every 8 connection(s)
I20260811 14:39:45.739648 4572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:45.740773 4686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:45.742501 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 4572
I20260811 14:39:45.742667 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:45.742899 2461 external_mini_cluster.cc:1428] Setting key ea411c2fa988aaddd69253fbfeff8864
I20260811 14:39:45.743851 4686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31: Bootstrap starting.
I20260811 14:39:45.744513 4686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:45.744918 4686 log.cc:824] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:45.745970 4686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31: No bootstrap required, opened a new log
I20260811 14:39:45.748549 4686 raft_consensus.cc:348] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25bb9c331f14f5bb2cab0569bd25d31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45373 } }
I20260811 14:39:45.748751 4686 raft_consensus.cc:374] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:45.748806 4686 raft_consensus.cc:729] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e25bb9c331f14f5bb2cab0569bd25d31, State: Initialized, Role: FOLLOWER
I20260811 14:39:45.748980 4686 consensus_queue.cc:260] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25bb9c331f14f5bb2cab0569bd25d31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45373 } }
I20260811 14:39:45.749083 4686 raft_consensus.cc:388] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:45.749122 4686 raft_consensus.cc:482] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:45.749174 4686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:45.750276 4686 raft_consensus.cc:504] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25bb9c331f14f5bb2cab0569bd25d31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45373 } }
I20260811 14:39:45.750442 4686 leader_election.cc:302] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e25bb9c331f14f5bb2cab0569bd25d31; no voters:
I20260811 14:39:45.750705 4686 leader_election.cc:288] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:45.750741 4694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:45.750906 4694 raft_consensus.cc:686] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31 [term 1 LEADER]: Becoming Leader. State: Replica: e25bb9c331f14f5bb2cab0569bd25d31, State: Running, Role: LEADER
I20260811 14:39:45.751044 4686 sys_catalog.cc:564] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:45.751067 4694 consensus_queue.cc:237] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25bb9c331f14f5bb2cab0569bd25d31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45373 } }
Aug 11 14:39:45 dist-test-slave-twwt krb5kdc[4549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 14:39:45.751437 4694 sys_catalog.cc:455] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e25bb9c331f14f5bb2cab0569bd25d31. Latest consensus state: current_term: 1 leader_uuid: "e25bb9c331f14f5bb2cab0569bd25d31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25bb9c331f14f5bb2cab0569bd25d31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45373 } } }
I20260811 14:39:45.751547 4694 sys_catalog.cc:458] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:45.751453 4695 sys_catalog.cc:455] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e25bb9c331f14f5bb2cab0569bd25d31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25bb9c331f14f5bb2cab0569bd25d31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45373 } } }
I20260811 14:39:45.751742 4695 sys_catalog.cc:458] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:45.751801 4701 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:45.752418 4701 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:45.755487 4714 mvcc.cc:205] Tried to move back new op lower bound from 7317336824847286272 to 7317336824836845568. Current Snapshot: MvccSnapshot[applied={T|T < 7317336824847286272}]
I20260811 14:39:45.756057 4701 catalog_manager.cc:1347] Generated new cluster ID: f81524673fab48578b8bccde5b13ff18
I20260811 14:39:45.756223 4701 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:39:45.757241 4690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:45.743592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:51407 (local address 127.0.0.1:45373)
0811 14:39:45.743798 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:45.743807 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:45.743836 (+ 29us) server_negotiation.cc:407] Connection header received
0811 14:39:45.744372 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:45.744381 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:45.744440 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:45.744566 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:45.745455 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:45.746489 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:45.747676 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:45.748134 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:45.748231 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:45.752036 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:45.752040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:45.752041 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:45.752055 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:45.754369 (+ 2314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:45.755026 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:45.755030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:45.755030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:45.755161 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:45.755627 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:45.755629 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:45.755629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:45.756018 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:45.756201 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:45.756784 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 14:39:45.756914 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":101,"threads_started":1}
I20260811 14:39:45.758450 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:39:45.769059 4690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:45.760232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:52053 (local address 127.0.0.1:45373)
0811 14:39:45.760277 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:45.760279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:45.760448 (+ 169us) server_negotiation.cc:407] Connection header received
0811 14:39:45.760528 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:45.760529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:45.760541 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:45.760597 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:45.761432 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:45.762342 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:45.763624 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:45.763983 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:45.764063 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:45.765819 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:45.765821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:45.765821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:45.765830 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:45.767613 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:45.768139 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:45.768141 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:45.768141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:45.768210 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:45.768475 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:45.768476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:45.768476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:45.768609 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:45.768713 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:45.768884 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 14:39:45.768958 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:39:45.782068 4701 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:45.782883 4701 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:45.785666 4701 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e25bb9c331f14f5bb2cab0569bd25d31: Generated new TSK 0
I20260811 14:39:45.785969 4701 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:39:45.791755 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 4572
W20260811 14:39:45.811072 4546 connection.cc:535] client connection to 127.0.0.1:45373 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45373): Connection reset by peer (error 104)
2026-08-11T14:39:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1632 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:39:45 dist-test-slave-twwt krb5kdc[4724](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:39:45 dist-test-slave-twwt krb5kdc[4724](info): set up 2 sockets
Aug 11 14:39:45 dist-test-slave-twwt krb5kdc[4724](info): commencing operation
krb5kdc: starting...
W20260811 14:39:47.153239 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.327s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:39:47 dist-test-slave-twwt krb5kdc[4724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:39:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:39:47.299788 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39575
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:47.388811 4744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:47.389097 4744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:47.389142 4744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:47.390391 4744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:47.390471 4744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:47.390498 4744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:47.390528 4744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:47.390559 4744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:47.392217 4744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:47.392529 4744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:47.392822 4744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:47.398330 4757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:47.398324 4754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:47.398324 4752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:47.398455 4744 server_base.cc:1034] running on GCE node
I20260811 14:39:47.398918 4744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:47.399276 4744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:47.400444 4744 hybrid_clock.cc:648] HybridClock initialized: now 1786459187400421 us; error 39 us; skew 500 ppm
Aug 11 14:39:47 dist-test-slave-twwt krb5kdc[4724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459187, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:47.403457 4744 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:39:47.404026 4744 webserver.cc:460] Webserver started at http://127.0.0.1:38693/ using document root <none> and password file <none>
I20260811 14:39:47.404338 4744 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:47.404423 4744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:47.404597 4744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:47.405879 4744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "36656e96048d4ff78c7a464e5bdcdbdd"
format_stamp: "Formatted at 2026-08-11 14:39:47 on dist-test-slave-twwt"
server_key: "2d0cccebda4dd354105f14bdcf9b745f"
server_key_iv: "4a6eff58c8f7fe9cb5689d778d1d6141"
server_key_version: "encryptionkey@0"
I20260811 14:39:47.406320 4744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "36656e96048d4ff78c7a464e5bdcdbdd"
format_stamp: "Formatted at 2026-08-11 14:39:47 on dist-test-slave-twwt"
server_key: "2d0cccebda4dd354105f14bdcf9b745f"
server_key_iv: "4a6eff58c8f7fe9cb5689d778d1d6141"
server_key_version: "encryptionkey@0"
I20260811 14:39:47.408025 4744 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260811 14:39:47.409122 4769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:47.409403 4744 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:47.409533 4744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "36656e96048d4ff78c7a464e5bdcdbdd"
format_stamp: "Formatted at 2026-08-11 14:39:47 on dist-test-slave-twwt"
server_key: "2d0cccebda4dd354105f14bdcf9b745f"
server_key_iv: "4a6eff58c8f7fe9cb5689d778d1d6141"
server_key_version: "encryptionkey@0"
I20260811 14:39:47.409621 4744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:47.439759 4744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:47.442162 4744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:47.442361 4744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:47.446738 4744 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39575
I20260811 14:39:47.446784 4861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39575 every 8 connection(s)
I20260811 14:39:47.447191 4744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:47.448246 4864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:47.450680 4864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd: Bootstrap starting.
I20260811 14:39:47.451298 4864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:47.451597 4864 log.cc:824] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:47.452378 4864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd: No bootstrap required, opened a new log
I20260811 14:39:47.454005 4864 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36656e96048d4ff78c7a464e5bdcdbdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39575 } }
I20260811 14:39:47.454154 4864 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:47.454177 4864 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36656e96048d4ff78c7a464e5bdcdbdd, State: Initialized, Role: FOLLOWER
I20260811 14:39:47.454344 4864 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36656e96048d4ff78c7a464e5bdcdbdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39575 } }
I20260811 14:39:47.454413 4864 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:47.454437 4864 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:47.454509 4864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:47.455099 4864 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36656e96048d4ff78c7a464e5bdcdbdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39575 } }
I20260811 14:39:47.455219 4864 leader_election.cc:302] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36656e96048d4ff78c7a464e5bdcdbdd; no voters:
I20260811 14:39:47.455417 4864 leader_election.cc:288] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:47.455472 4868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:47.455696 4864 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:47.455705 4868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd [term 1 LEADER]: Becoming Leader. State: Replica: 36656e96048d4ff78c7a464e5bdcdbdd, State: Running, Role: LEADER
I20260811 14:39:47.455914 4868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36656e96048d4ff78c7a464e5bdcdbdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39575 } }
I20260811 14:39:47.456516 4873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36656e96048d4ff78c7a464e5bdcdbdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36656e96048d4ff78c7a464e5bdcdbdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39575 } } }
I20260811 14:39:47.456626 4873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:47.457064 4874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36656e96048d4ff78c7a464e5bdcdbdd. Latest consensus state: current_term: 1 leader_uuid: "36656e96048d4ff78c7a464e5bdcdbdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36656e96048d4ff78c7a464e5bdcdbdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39575 } } }
I20260811 14:39:47.457175 4874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:47.457116 4885 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:47.457325 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 4744
I20260811 14:39:47.457471 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:47.457696 2461 external_mini_cluster.cc:1428] Setting key 0726e6c1f067f97e3a753e97e5b15e75
I20260811 14:39:47.457841 4885 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:47.459887 4885 catalog_manager.cc:1347] Generated new cluster ID: 3c774ff2732a41aa8035de84e785c92f
I20260811 14:39:47.459949 4885 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:39:47.465401 4899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:47.458448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:42667 (local address 127.0.0.1:39575)
0811 14:39:47.458675 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:47.458685 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:47.458718 (+ 33us) server_negotiation.cc:407] Connection header received
0811 14:39:47.458848 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:47.458855 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:47.458918 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:47.459075 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:47.459552 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:47.460989 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:47.462588 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:47.463336 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:47.463462 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:47.464415 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:47.464427 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:47.464428 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:47.464667 (+ 239us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:47.464711 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:47.464728 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:47.464824 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:47.464962 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 14:39:47.464999 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":85,"threads_started":1}
I20260811 14:39:47.466552 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:39:47.475050 4899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:47.471073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:47925 (local address 127.0.0.1:39575)
0811 14:39:47.471128 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:47.471130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:47.471249 (+ 119us) server_negotiation.cc:407] Connection header received
0811 14:39:47.471301 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:47.471302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:47.471316 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:47.471384 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:47.471539 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:47.472372 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:47.473472 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:47.473833 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:47.473902 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:47.474745 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:47.474746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:47.474747 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:47.474753 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:47.474771 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:47.474776 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:47.474851 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:47.474935 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 14:39:47.474950 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 14:39:47.477082 4885 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:47.477633 4885 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:47.482342 4885 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36656e96048d4ff78c7a464e5bdcdbdd: Generated new TSK 0
I20260811 14:39:47.482558 4885 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:39:47.490607 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 4744
W20260811 14:39:47.503778 4722 connection.cc:535] client connection to 127.0.0.1:39575 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39575): Connection reset by peer (error 104)
2026-08-11T14:39:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1691 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:39:47 dist-test-slave-twwt krb5kdc[4915](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:39:47 dist-test-slave-twwt krb5kdc[4915](info): set up 2 sockets
Aug 11 14:39:47 dist-test-slave-twwt krb5kdc[4915](info): commencing operation
krb5kdc: starting...
W20260811 14:39:48.779227 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:39:48 dist-test-slave-twwt krb5kdc[4915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:39:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:39:48.911984 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40067
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:48.998140 4938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:48.998397 4938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:48.998441 4938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:48.999717 4938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:48.999804 4938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:48.999831 4938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:48.999862 4938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:48.999890 4938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:49.001477 4938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:49.001791 4938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:49.002058 4938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:49.007124 4952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:49.007117 4944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:49.007117 4946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:49.007603 4938 server_base.cc:1034] running on GCE node
I20260811 14:39:49.007879 4938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:49.008184 4938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:49.009371 4938 hybrid_clock.cc:648] HybridClock initialized: now 1786459189009355 us; error 45 us; skew 500 ppm
Aug 11 14:39:49 dist-test-slave-twwt krb5kdc[4915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459189, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:49.012161 4938 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:39:49.012656 4938 webserver.cc:460] Webserver started at http://127.0.0.1:46521/ using document root <none> and password file <none>
I20260811 14:39:49.012953 4938 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:49.013038 4938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:49.013195 4938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:49.014330 4938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "301a8b8630d14e7e9ad66c354b7ed834"
format_stamp: "Formatted at 2026-08-11 14:39:49 on dist-test-slave-twwt"
server_key: "6ede95e8c59783412151de3fbe6201bb"
server_key_iv: "23fa9de1d9ac7ca789641aa5c391603c"
server_key_version: "encryptionkey@0"
I20260811 14:39:49.014772 4938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "301a8b8630d14e7e9ad66c354b7ed834"
format_stamp: "Formatted at 2026-08-11 14:39:49 on dist-test-slave-twwt"
server_key: "6ede95e8c59783412151de3fbe6201bb"
server_key_iv: "23fa9de1d9ac7ca789641aa5c391603c"
server_key_version: "encryptionkey@0"
I20260811 14:39:49.016220 4938 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:39:49.017086 4961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:49.017381 4938 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:39:49.017498 4938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "301a8b8630d14e7e9ad66c354b7ed834"
format_stamp: "Formatted at 2026-08-11 14:39:49 on dist-test-slave-twwt"
server_key: "6ede95e8c59783412151de3fbe6201bb"
server_key_iv: "23fa9de1d9ac7ca789641aa5c391603c"
server_key_version: "encryptionkey@0"
I20260811 14:39:49.017587 4938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:49.021121 4938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:49.023411 4938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:49.023573 4938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:49.027900 4938 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40067
I20260811 14:39:49.027976 5041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40067 every 8 connection(s)
I20260811 14:39:49.028334 4938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:49.029386 5046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:49.032074 5046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834: Bootstrap starting.
I20260811 14:39:49.032723 5046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:49.033030 5046 log.cc:824] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:49.033768 5046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834: No bootstrap required, opened a new log
I20260811 14:39:49.035539 5046 raft_consensus.cc:348] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "301a8b8630d14e7e9ad66c354b7ed834" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40067 } }
I20260811 14:39:49.035687 5046 raft_consensus.cc:374] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:49.035709 5046 raft_consensus.cc:729] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301a8b8630d14e7e9ad66c354b7ed834, State: Initialized, Role: FOLLOWER
I20260811 14:39:49.035874 5046 consensus_queue.cc:260] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "301a8b8630d14e7e9ad66c354b7ed834" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40067 } }
I20260811 14:39:49.035946 5046 raft_consensus.cc:388] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:49.035969 5046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:49.036021 5046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:49.036631 5046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "301a8b8630d14e7e9ad66c354b7ed834" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40067 } }
I20260811 14:39:49.036748 5046 leader_election.cc:302] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 301a8b8630d14e7e9ad66c354b7ed834; no voters:
I20260811 14:39:49.036944 5046 leader_election.cc:288] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:49.037017 5050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:49.037233 5046 sys_catalog.cc:564] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:49.037225 5050 raft_consensus.cc:686] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834 [term 1 LEADER]: Becoming Leader. State: Replica: 301a8b8630d14e7e9ad66c354b7ed834, State: Running, Role: LEADER
I20260811 14:39:49.037463 5050 consensus_queue.cc:237] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "301a8b8630d14e7e9ad66c354b7ed834" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40067 } }
I20260811 14:39:49.038036 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 4938
I20260811 14:39:49.038167 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:49.038313 5051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "301a8b8630d14e7e9ad66c354b7ed834" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "301a8b8630d14e7e9ad66c354b7ed834" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40067 } } }
I20260811 14:39:49.038403 2461 external_mini_cluster.cc:1428] Setting key 44f4bfc2efbda96b0b7bf41594482b91
I20260811 14:39:49.038412 5051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834 [sys.catalog]: This master's current role is: LEADER
W20260811 14:39:49.039373 5082 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260811 14:39:49.038822 5052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 301a8b8630d14e7e9ad66c354b7ed834. Latest consensus state: current_term: 1 leader_uuid: "301a8b8630d14e7e9ad66c354b7ed834" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "301a8b8630d14e7e9ad66c354b7ed834" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40067 } } }
W20260811 14:39:49.039446 5082 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:39:49.039461 5052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:49.039510 5072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:49.039686 5072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:49.041242 5072 catalog_manager.cc:1347] Generated new cluster ID: 6c0505d342e6480f84c99c54c7ec5d85
I20260811 14:39:49.041301 5072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:39:49.042043 5086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:49.039166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:59303 (local address 127.0.0.1:40067)
0811 14:39:49.040245 (+ 1079us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:49.040253 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:49.040279 (+ 26us) server_negotiation.cc:407] Connection header received
0811 14:39:49.040382 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:49.040387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:49.040412 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:49.040744 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:49.041638 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:49.041641 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:49.041642 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:49.041645 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:49.041683 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:49.041694 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:49.041762 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:49.041842 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 14:39:49.041890 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1033,"thread_start_us":88,"threads_started":1}
I20260811 14:39:49.043079 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:39:49.045279 5086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:49.044418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:54269 (local address 127.0.0.1:40067)
0811 14:39:49.044453 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:49.044455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:49.044526 (+ 71us) server_negotiation.cc:407] Connection header received
0811 14:39:49.044593 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:49.044594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:49.044599 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:49.044646 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:49.045116 (+ 470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:49.045117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:49.045117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:49.045120 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:49.045132 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:49.045136 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:49.045179 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:49.045225 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 14:39:49.045236 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 14:39:49.057976 5072 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:49.058175 5072 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:49.062116 5072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 301a8b8630d14e7e9ad66c354b7ed834: Generated new TSK 0
I20260811 14:39:49.062325 5072 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:39:49.072476 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 4938
2026-08-11T14:39:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1589 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:39:49 dist-test-slave-twwt krb5kdc[5102](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:39:49 dist-test-slave-twwt krb5kdc[5102](info): set up 2 sockets
Aug 11 14:39:49 dist-test-slave-twwt krb5kdc[5102](info): commencing operation
krb5kdc: starting...
W20260811 14:39:50.386994 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:39:50 dist-test-slave-twwt krb5kdc[5102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:39:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:39:50.519197 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43249
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:50.607293 5119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:50.607597 5119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:50.607643 5119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:50.608880 5119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:50.608918 5119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:50.608934 5119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:50.608947 5119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:50.608959 5119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:50.610451 5119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43249
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:50.610736 5119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:50.610953 5119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:50.616178 5127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:50.616185 5129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:50.616187 5132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:50.616480 5119 server_base.cc:1034] running on GCE node
I20260811 14:39:50.616736 5119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:50.617043 5119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:50.618238 5119 hybrid_clock.cc:648] HybridClock initialized: now 1786459190618222 us; error 33 us; skew 500 ppm
Aug 11 14:39:50 dist-test-slave-twwt krb5kdc[5102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459190, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:50.621178 5119 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:39:50.621794 5119 webserver.cc:460] Webserver started at http://127.0.0.1:40243/ using document root <none> and password file <none>
I20260811 14:39:50.622059 5119 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:50.622138 5119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:50.622298 5119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:50.623528 5119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "1dced8450dcc4680994f232d0bbada22"
format_stamp: "Formatted at 2026-08-11 14:39:50 on dist-test-slave-twwt"
server_key: "5f01e372211ac7aab369c2d53e6a28ad"
server_key_iv: "c0ab332626527f12c7eab5209f88dd71"
server_key_version: "encryptionkey@0"
I20260811 14:39:50.623927 5119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "1dced8450dcc4680994f232d0bbada22"
format_stamp: "Formatted at 2026-08-11 14:39:50 on dist-test-slave-twwt"
server_key: "5f01e372211ac7aab369c2d53e6a28ad"
server_key_iv: "c0ab332626527f12c7eab5209f88dd71"
server_key_version: "encryptionkey@0"
I20260811 14:39:50.625392 5119 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:39:50.626268 5141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:50.626474 5119 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:50.626595 5119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "1dced8450dcc4680994f232d0bbada22"
format_stamp: "Formatted at 2026-08-11 14:39:50 on dist-test-slave-twwt"
server_key: "5f01e372211ac7aab369c2d53e6a28ad"
server_key_iv: "c0ab332626527f12c7eab5209f88dd71"
server_key_version: "encryptionkey@0"
I20260811 14:39:50.626783 5119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:50.651077 5119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:50.653646 5119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:50.653834 5119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:50.658584 5119 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43249
I20260811 14:39:50.658691 5221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43249 every 8 connection(s)
I20260811 14:39:50.659089 5119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:50.660156 5223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:50.662637 5223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22: Bootstrap starting.
I20260811 14:39:50.663323 5223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:50.663645 5223 log.cc:824] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:50.664525 5223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22: No bootstrap required, opened a new log
I20260811 14:39:50.665628 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 5119
I20260811 14:39:50.665769 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:50.665987 2461 external_mini_cluster.cc:1428] Setting key 752bc9580b30ed809943e8ff14400287
I20260811 14:39:50.666945 5223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dced8450dcc4680994f232d0bbada22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43249 } }
I20260811 14:39:50.667104 5223 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:50.667126 5223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dced8450dcc4680994f232d0bbada22, State: Initialized, Role: FOLLOWER
I20260811 14:39:50.667418 5223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dced8450dcc4680994f232d0bbada22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43249 } }
I20260811 14:39:50.667552 5223 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:50.667608 5223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:50.667675 5223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:50.668861 5223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dced8450dcc4680994f232d0bbada22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43249 } }
I20260811 14:39:50.669011 5223 leader_election.cc:302] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1dced8450dcc4680994f232d0bbada22; no voters:
I20260811 14:39:50.669272 5223 leader_election.cc:288] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:50.669299 5233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:50.669433 5233 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22 [term 1 LEADER]: Becoming Leader. State: Replica: 1dced8450dcc4680994f232d0bbada22, State: Running, Role: LEADER
I20260811 14:39:50.669629 5223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:50.669600 5233 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dced8450dcc4680994f232d0bbada22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43249 } }
I20260811 14:39:50.670172 5235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dced8450dcc4680994f232d0bbada22. Latest consensus state: current_term: 1 leader_uuid: "1dced8450dcc4680994f232d0bbada22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dced8450dcc4680994f232d0bbada22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43249 } } }
I20260811 14:39:50.670291 5235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:50.670531 5234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dced8450dcc4680994f232d0bbada22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dced8450dcc4680994f232d0bbada22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43249 } } }
I20260811 14:39:50.670697 5234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:50.670913 5243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:50.671702 5243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:50.673528 5243 catalog_manager.cc:1347] Generated new cluster ID: 4e41a31810f54ce4bbf4558993542feb
I20260811 14:39:50.673590 5243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:39:50 dist-test-slave-twwt krb5kdc[5102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 14:39:50.679323 5229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:50.666600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:51857 (local address 127.0.0.1:43249)
0811 14:39:50.666909 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:50.666918 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:50.666948 (+ 30us) server_negotiation.cc:407] Connection header received
0811 14:39:50.667059 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:50.667065 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:50.667146 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:50.667328 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:50.668213 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:50.669333 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:50.670418 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:50.671193 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:50.671329 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:50.674844 (+ 3515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:50.674850 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:50.674850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:50.674868 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:50.677140 (+ 2272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:50.677635 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:50.677639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:50.677640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:50.677811 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:50.678082 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:50.678083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:50.678083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:50.678443 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:50.678662 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:50.678767 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 14:39:50.679036 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":139,"threads_started":1}
I20260811 14:39:50.681151 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:39:50.690783 5243 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:50.691550 5243 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:50.691807 5229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:50.682926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:37739 (local address 127.0.0.1:43249)
0811 14:39:50.682972 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:50.682974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:50.683124 (+ 150us) server_negotiation.cc:407] Connection header received
0811 14:39:50.683239 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:50.683240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:50.683251 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:50.683286 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:50.684148 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:50.685009 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:50.686238 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:50.686599 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:50.686710 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:50.688486 (+ 1776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:50.688488 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:50.688488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:50.688497 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:50.690284 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:50.690857 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:50.690858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:50.690858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:50.690927 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:50.691186 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:50.691187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:50.691188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:50.691333 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:50.691463 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:50.691613 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 14:39:50.691691 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 14:39:50.695510 5243 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dced8450dcc4680994f232d0bbada22: Generated new TSK 0
I20260811 14:39:50.695713 5243 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:39:50.699739 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 5119
W20260811 14:39:50.712726 5099 connection.cc:535] client connection to 127.0.0.1:43249 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43249): Connection reset by peer (error 104)
2026-08-11T14:39:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1619 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:39:50 dist-test-slave-twwt krb5kdc[5276](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:39:50 dist-test-slave-twwt krb5kdc[5276](info): set up 2 sockets
Aug 11 14:39:50 dist-test-slave-twwt krb5kdc[5276](info): commencing operation
krb5kdc: starting...
W20260811 14:39:51.971947 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:39:52 dist-test-slave-twwt krb5kdc[5276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:39:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:39:52.105628 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38269
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:52.192312 5296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:52.192528 5296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:52.192551 5296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:52.193686 5296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:52.193725 5296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:52.193740 5296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:52.193753 5296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:52.193764 5296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:52.195447 5296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38269
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:52.195667 5296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:52.195856 5296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:52.200985 5305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:52.200974 5308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:52.201048 5306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:52.201298 5296 server_base.cc:1034] running on GCE node
I20260811 14:39:52.201550 5296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:52.201931 5296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:52.203110 5296 hybrid_clock.cc:648] HybridClock initialized: now 1786459192203110 us; error 46 us; skew 500 ppm
Aug 11 14:39:52 dist-test-slave-twwt krb5kdc[5276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459192, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:52.205931 5296 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:39:52.206475 5296 webserver.cc:460] Webserver started at http://127.0.0.1:39607/ using document root <none> and password file <none>
I20260811 14:39:52.206807 5296 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:52.206882 5296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:52.207034 5296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:52.208185 5296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "5bf82373ff8c401c80883564b685d2f4"
format_stamp: "Formatted at 2026-08-11 14:39:52 on dist-test-slave-twwt"
server_key: "7bf413c7a594a395dd3ef92eacf22bae"
server_key_iv: "bc336e67ce2026a8d36ac138ee510c58"
server_key_version: "encryptionkey@0"
I20260811 14:39:52.208577 5296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "5bf82373ff8c401c80883564b685d2f4"
format_stamp: "Formatted at 2026-08-11 14:39:52 on dist-test-slave-twwt"
server_key: "7bf413c7a594a395dd3ef92eacf22bae"
server_key_iv: "bc336e67ce2026a8d36ac138ee510c58"
server_key_version: "encryptionkey@0"
I20260811 14:39:52.210019 5296 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:39:52.210913 5318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:52.211088 5296 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:52.211201 5296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "5bf82373ff8c401c80883564b685d2f4"
format_stamp: "Formatted at 2026-08-11 14:39:52 on dist-test-slave-twwt"
server_key: "7bf413c7a594a395dd3ef92eacf22bae"
server_key_iv: "bc336e67ce2026a8d36ac138ee510c58"
server_key_version: "encryptionkey@0"
I20260811 14:39:52.211278 5296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:52.225188 5296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:52.227737 5296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:52.227926 5296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:52.232733 5296 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38269
I20260811 14:39:52.232802 5404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38269 every 8 connection(s)
I20260811 14:39:52.233237 5296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:52.234161 5405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:52.236629 5405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4: Bootstrap starting.
I20260811 14:39:52.237263 5405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:52.237568 5405 log.cc:824] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:52.238469 5405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4: No bootstrap required, opened a new log
I20260811 14:39:52.240260 5405 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bf82373ff8c401c80883564b685d2f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38269 } }
I20260811 14:39:52.240410 5405 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:52.240434 5405 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf82373ff8c401c80883564b685d2f4, State: Initialized, Role: FOLLOWER
I20260811 14:39:52.240604 5405 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bf82373ff8c401c80883564b685d2f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38269 } }
I20260811 14:39:52.240669 5405 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:52.240713 5405 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:52.240767 5405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:52.241353 5405 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bf82373ff8c401c80883564b685d2f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38269 } }
I20260811 14:39:52.241449 5405 leader_election.cc:302] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bf82373ff8c401c80883564b685d2f4; no voters:
I20260811 14:39:52.241647 5405 leader_election.cc:288] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:52.241696 5409 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:52.241967 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 5296
I20260811 14:39:52.242123 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:52.242343 2461 external_mini_cluster.cc:1428] Setting key 51de39ed8fbe89bff714d30486d80184
I20260811 14:39:52.241955 5405 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:52.242689 5409 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4 [term 1 LEADER]: Becoming Leader. State: Replica: 5bf82373ff8c401c80883564b685d2f4, State: Running, Role: LEADER
I20260811 14:39:52.242856 5409 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bf82373ff8c401c80883564b685d2f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38269 } }
I20260811 14:39:52.243423 5413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bf82373ff8c401c80883564b685d2f4. Latest consensus state: current_term: 1 leader_uuid: "5bf82373ff8c401c80883564b685d2f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bf82373ff8c401c80883564b685d2f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38269 } } }
I20260811 14:39:52.243520 5413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:52.243700 5411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bf82373ff8c401c80883564b685d2f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bf82373ff8c401c80883564b685d2f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38269 } } }
I20260811 14:39:52.243824 5411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:52.244299 5426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:52.244953 5426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:52.247812 5426 catalog_manager.cc:1347] Generated new cluster ID: a48c579930a44ba5b5e7bacb13c30fa4
I20260811 14:39:52.247923 5426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:39:52.248903 5423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:52.243190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:47041 (local address 127.0.0.1:38269)
0811 14:39:52.243675 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:52.243686 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:52.243722 (+ 36us) server_negotiation.cc:407] Connection header received
0811 14:39:52.243874 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:52.243884 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:52.243949 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:52.244117 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:52.244429 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:52.245548 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:52.246732 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:52.247218 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:52.247325 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:52.248215 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:52.248219 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:52.248219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:52.248428 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:52.248465 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:52.248477 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:52.248576 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:52.248667 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 14:39:52.248699 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260811 14:39:52.250102 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:39:52.254344 5426 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:52.255129 5426 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:52.261761 5423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:52.254171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:47965 (local address 127.0.0.1:38269)
0811 14:39:52.254217 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:52.254220 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:52.254682 (+ 462us) server_negotiation.cc:407] Connection header received
0811 14:39:52.258831 (+ 4149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:52.258832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:52.258846 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:52.258907 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:52.259090 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:52.259618 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:52.260442 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:52.260671 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:52.260749 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:52.261496 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:52.261498 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:52.261498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:52.261504 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:52.261520 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:52.261525 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:52.261583 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:52.261660 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 14:39:52.261672 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":22}
I20260811 14:39:52.275596 5426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bf82373ff8c401c80883564b685d2f4: Generated new TSK 0
I20260811 14:39:52.275859 5426 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:39:52.283937 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 5296
W20260811 14:39:52.303481 5273 connection.cc:535] client connection to 127.0.0.1:38269 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38269): Connection reset by peer (error 104)
2026-08-11T14:39:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1590 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:39:52 dist-test-slave-twwt krb5kdc[5459](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:39:52 dist-test-slave-twwt krb5kdc[5459](info): set up 2 sockets
Aug 11 14:39:52 dist-test-slave-twwt krb5kdc[5459](info): commencing operation
krb5kdc: starting...
W20260811 14:39:53.601662 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:39:53 dist-test-slave-twwt krb5kdc[5459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:39:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:39:53.737231 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40749
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:53.824824 5485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:53.825086 5485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:53.825129 5485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:53.826333 5485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:53.826411 5485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:53.826434 5485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:53.826455 5485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:53.826486 5485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:53.828112 5485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40749
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:53.828429 5485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:53.828679 5485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:53.833937 5492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:53.833998 5493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:53.834028 5485 server_base.cc:1034] running on GCE node
W20260811 14:39:53.834062 5498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:53.834461 5485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:53.834823 5485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:53.836024 5485 hybrid_clock.cc:648] HybridClock initialized: now 1786459193836002 us; error 41 us; skew 500 ppm
Aug 11 14:39:53 dist-test-slave-twwt krb5kdc[5459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459193, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:53.844601 5485 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:39:53.845171 5485 webserver.cc:460] Webserver started at http://127.0.0.1:45999/ using document root <none> and password file <none>
I20260811 14:39:53.845451 5485 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:53.845499 5485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:53.845661 5485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:53.846890 5485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "ebf22276035344668fc48763dbf73826"
format_stamp: "Formatted at 2026-08-11 14:39:53 on dist-test-slave-twwt"
server_key: "cc82627c9f967ed0f58ba29f0f501929"
server_key_iv: "36aa5eafc84bd243f9545791c851f283"
server_key_version: "encryptionkey@0"
I20260811 14:39:53.847270 5485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "ebf22276035344668fc48763dbf73826"
format_stamp: "Formatted at 2026-08-11 14:39:53 on dist-test-slave-twwt"
server_key: "cc82627c9f967ed0f58ba29f0f501929"
server_key_iv: "36aa5eafc84bd243f9545791c851f283"
server_key_version: "encryptionkey@0"
I20260811 14:39:53.848718 5485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:53.849553 5515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:53.849893 5485 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:53.849987 5485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "ebf22276035344668fc48763dbf73826"
format_stamp: "Formatted at 2026-08-11 14:39:53 on dist-test-slave-twwt"
server_key: "cc82627c9f967ed0f58ba29f0f501929"
server_key_iv: "36aa5eafc84bd243f9545791c851f283"
server_key_version: "encryptionkey@0"
I20260811 14:39:53.850095 5485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:53.853734 5485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:53.856178 5485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:53.856319 5485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:53.860622 5485 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40749
I20260811 14:39:53.860680 5591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40749 every 8 connection(s)
I20260811 14:39:53.861032 5485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:53.861931 5592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:53.863740 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 5485
I20260811 14:39:53.863883 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:53.864073 2461 external_mini_cluster.cc:1428] Setting key e6a84856b5bc54fadfa188b5257a3303
I20260811 14:39:53.864677 5592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826: Bootstrap starting.
I20260811 14:39:53.865442 5592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:53.865789 5592 log.cc:824] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:53.866557 5592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826: No bootstrap required, opened a new log
I20260811 14:39:53.866942 5599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:53.864627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:50233 (local address 127.0.0.1:40749)
0811 14:39:53.864870 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:53.864889 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:53.864926 (+ 37us) server_negotiation.cc:407] Connection header received
0811 14:39:53.865552 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:53.865563 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:53.865606 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:53.865785 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:53.866501 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:53.866505 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:53.866505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:53.866510 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:53.866558 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:53.866568 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:53.866687 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:53.866726 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 14:39:53.866771 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":88,"threads_started":1}
I20260811 14:39:53.867981 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:39:53.869040 5592 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebf22276035344668fc48763dbf73826" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40749 } }
I20260811 14:39:53.869197 5592 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:53.869251 5592 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebf22276035344668fc48763dbf73826, State: Initialized, Role: FOLLOWER
I20260811 14:39:53.869376 5592 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebf22276035344668fc48763dbf73826" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40749 } }
I20260811 14:39:53.869499 5592 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:53.869545 5592 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:53.869606 5592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:53.870437 5592 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebf22276035344668fc48763dbf73826" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40749 } }
I20260811 14:39:53.870565 5592 leader_election.cc:302] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebf22276035344668fc48763dbf73826; no voters:
I20260811 14:39:53.870656 5599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:53.869412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:49685 (local address 127.0.0.1:40749)
0811 14:39:53.869463 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:53.869465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:53.869640 (+ 175us) server_negotiation.cc:407] Connection header received
0811 14:39:53.869711 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:53.869711 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:53.869716 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:53.869779 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:53.870336 (+ 557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:53.870337 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:53.870338 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:53.870341 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:53.870353 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:53.870357 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:53.870405 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:53.870491 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 14:39:53.870502 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 14:39:53.870824 5592 leader_election.cc:288] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:53.871107 5609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:53.871321 5609 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826 [term 1 LEADER]: Becoming Leader. State: Replica: ebf22276035344668fc48763dbf73826, State: Running, Role: LEADER
I20260811 14:39:53.871343 5592 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:53.871423 5609 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebf22276035344668fc48763dbf73826" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40749 } }
I20260811 14:39:53.871796 5610 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebf22276035344668fc48763dbf73826" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebf22276035344668fc48763dbf73826" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40749 } } }
I20260811 14:39:53.871821 5611 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebf22276035344668fc48763dbf73826. Latest consensus state: current_term: 1 leader_uuid: "ebf22276035344668fc48763dbf73826" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebf22276035344668fc48763dbf73826" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40749 } } }
I20260811 14:39:53.871899 5610 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:53.871905 5611 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:53.872159 5619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:53.872764 5619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:53.874346 5619 catalog_manager.cc:1347] Generated new cluster ID: ff83d6b3519d46afa6a61caa8f2a5d85
I20260811 14:39:53.874410 5619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:39:53.886931 5619 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:53.887138 5619 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:53.892961 5619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebf22276035344668fc48763dbf73826: Generated new TSK 0
I20260811 14:39:53.893174 5619 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:39:53.901873 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 5485
2026-08-11T14:39:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1614 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:39:53 dist-test-slave-twwt krb5kdc[5637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:39:53 dist-test-slave-twwt krb5kdc[5637](info): set up 2 sockets
Aug 11 14:39:53 dist-test-slave-twwt krb5kdc[5637](info): commencing operation
krb5kdc: starting...
W20260811 14:39:55.176738 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:39:55 dist-test-slave-twwt krb5kdc[5637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:39:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:39:55.310262 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:42971
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:34831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:42971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:55.397617 5663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:55.397866 5663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:55.397917 5663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:55.399230 5663 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:55.399293 5663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:55.399317 5663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:55.399348 5663 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:55.399371 5663 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:55.400866 5663 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:34831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:42971
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:42971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:55.401156 5663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:55.401382 5663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:55.406487 5671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:55.406512 5675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:55.406538 5663 server_base.cc:1034] running on GCE node
W20260811 14:39:55.406499 5677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:55.407073 5663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:55.407373 5663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:55.408519 5663 hybrid_clock.cc:648] HybridClock initialized: now 1786459195408507 us; error 29 us; skew 500 ppm
Aug 11 14:39:55 dist-test-slave-twwt krb5kdc[5637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459195, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:55.411038 5663 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:39:55.411484 5663 webserver.cc:460] Webserver started at http://127.2.103.126:39429/ using document root <none> and password file <none>
I20260811 14:39:55.411708 5663 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:55.411754 5663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:55.411896 5663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:55.412981 5663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "5a25f24534aa4f03af25eb1ae2be9576"
format_stamp: "Formatted at 2026-08-11 14:39:55 on dist-test-slave-twwt"
server_key: "df6bd27628bd6e7867ea23ad3ffece63"
server_key_iv: "a96e5410f30803660b322134773a847b"
server_key_version: "encryptionkey@0"
I20260811 14:39:55.413352 5663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "5a25f24534aa4f03af25eb1ae2be9576"
format_stamp: "Formatted at 2026-08-11 14:39:55 on dist-test-slave-twwt"
server_key: "df6bd27628bd6e7867ea23ad3ffece63"
server_key_iv: "a96e5410f30803660b322134773a847b"
server_key_version: "encryptionkey@0"
I20260811 14:39:55.414779 5663 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 14:39:55.415618 5689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:55.415838 5663 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:55.415944 5663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "5a25f24534aa4f03af25eb1ae2be9576"
format_stamp: "Formatted at 2026-08-11 14:39:55 on dist-test-slave-twwt"
server_key: "df6bd27628bd6e7867ea23ad3ffece63"
server_key_iv: "a96e5410f30803660b322134773a847b"
server_key_version: "encryptionkey@0"
I20260811 14:39:55.416018 5663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:55.428232 5663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:55.430202 5663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:55.430380 5663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:55.434834 5663 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:42971
I20260811 14:39:55.434891 5773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:42971 every 8 connection(s)
I20260811 14:39:55.435300 5663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:55.436239 5774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:55.436529 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 5663
I20260811 14:39:55.436664 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:55.436858 2461 external_mini_cluster.cc:1428] Setting key f541f85c029744524dc0098715d4e449
I20260811 14:39:55.439452 5774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576: Bootstrap starting.
I20260811 14:39:55.440089 5774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:55.440399 5774 log.cc:824] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:55.441121 5774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576: No bootstrap required, opened a new log
I20260811 14:39:55.443437 5774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25f24534aa4f03af25eb1ae2be9576" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42971 } }
I20260811 14:39:55.443639 5774 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:55.443677 5774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a25f24534aa4f03af25eb1ae2be9576, State: Initialized, Role: FOLLOWER
Aug 11 14:39:55 dist-test-slave-twwt krb5kdc[5637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:39:55.443810 5774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25f24534aa4f03af25eb1ae2be9576" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42971 } }
I20260811 14:39:55.443910 5774 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:55.443944 5774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:55.443997 5774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:55.444731 5774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25f24534aa4f03af25eb1ae2be9576" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42971 } }
I20260811 14:39:55.444852 5774 leader_election.cc:302] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a25f24534aa4f03af25eb1ae2be9576; no voters:
I20260811 14:39:55.445029 5774 leader_election.cc:288] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:55.445093 5781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:55.445261 5781 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576 [term 1 LEADER]: Becoming Leader. State: Replica: 5a25f24534aa4f03af25eb1ae2be9576, State: Running, Role: LEADER
I20260811 14:39:55.445369 5774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:55.445398 5781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25f24534aa4f03af25eb1ae2be9576" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42971 } }
I20260811 14:39:55.445832 5781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a25f24534aa4f03af25eb1ae2be9576. Latest consensus state: current_term: 1 leader_uuid: "5a25f24534aa4f03af25eb1ae2be9576" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25f24534aa4f03af25eb1ae2be9576" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42971 } } }
I20260811 14:39:55.445953 5781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:55.445870 5785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a25f24534aa4f03af25eb1ae2be9576" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25f24534aa4f03af25eb1ae2be9576" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 42971 } } }
I20260811 14:39:55.446058 5785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:55.446398 5802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:55.447952 5802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:55.448544 5778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:55.437429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58369 (local address 127.2.103.126:42971)
0811 14:39:55.437624 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:55.437640 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:55.437667 (+ 27us) server_negotiation.cc:407] Connection header received
0811 14:39:55.438107 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:55.438114 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:55.438161 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:55.438281 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:55.439155 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:55.440100 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:55.440974 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:55.441437 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:55.441559 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:55.444398 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:55.444402 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:55.444402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:55.444414 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:55.446665 (+ 2251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:55.447216 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:55.447220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:55.447220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:55.447349 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:55.447626 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:55.447627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:55.447627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:55.447917 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:55.448113 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:55.448214 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 14:39:55.448326 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
I20260811 14:39:55.449597 5802 catalog_manager.cc:1347] Generated new cluster ID: 2aa790fb1d8c4c5696a32836af05cba7
I20260811 14:39:55.449654 5802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:39:55.449640 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:39:55.459322 5778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:55.451091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41039 (local address 127.2.103.126:42971)
0811 14:39:55.451127 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:55.451129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:55.451379 (+ 250us) server_negotiation.cc:407] Connection header received
0811 14:39:55.451482 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:55.451483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:55.451493 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:55.451524 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:55.452272 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:55.453046 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:55.454183 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:55.454549 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:55.454593 (+ 44us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:55.456228 (+ 1635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:55.456230 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:55.456230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:39:55.456237 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:55.457848 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:55.458314 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:55.458315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:55.458315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:55.458375 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:39:55.458658 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:39:55.458659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:39:55.458660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:39:55.458796 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:55.458882 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:55.459140 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 14:39:55.459212 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 14:39:55.464107 5802 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:55.464639 5802 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:55.469813 5802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a25f24534aa4f03af25eb1ae2be9576: Generated new TSK 0
I20260811 14:39:55.469992 5802 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:39:55.472149 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 5663
W20260811 14:39:55.490585 5635 connection.cc:535] client connection to 127.2.103.126:42971 recv error: Network error: failed to read from TLS socket (remote: 127.2.103.126:42971): Connection reset by peer (error 104)
2026-08-11T14:39:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1572 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:39:55 dist-test-slave-twwt krb5kdc[5833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:39:55 dist-test-slave-twwt krb5kdc[5833](info): set up 2 sockets
Aug 11 14:39:55 dist-test-slave-twwt krb5kdc[5833](info): commencing operation
krb5kdc: starting...
W20260811 14:39:56.762477 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:39:56 dist-test-slave-twwt krb5kdc[5833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:39:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:39:56.897945 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:33643
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:42411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:33643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:56.985186 5852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:56.985437 5852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:56.985463 5852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:56.986654 5852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:56.986692 5852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:56.986707 5852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:56.986720 5852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:56.986732 5852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:56.988315 5852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:42411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:33643
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:33643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:56.988541 5852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:56.988736 5852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:56.993824 5865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:56.993893 5852 server_base.cc:1034] running on GCE node
W20260811 14:39:56.993815 5866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:56.993840 5872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:56.994264 5852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:56.994575 5852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:56.995800 5852 hybrid_clock.cc:648] HybridClock initialized: now 1786459196995773 us; error 39 us; skew 500 ppm
Aug 11 14:39:56 dist-test-slave-twwt krb5kdc[5833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459196, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:56.998909 5852 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:39:56.999379 5852 webserver.cc:460] Webserver started at http://127.2.103.126:40805/ using document root <none> and password file <none>
I20260811 14:39:56.999614 5852 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:56.999661 5852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:56.999763 5852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:57.000967 5852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "7c16508723fb4777bf9b56a89e56bbf8"
format_stamp: "Formatted at 2026-08-11 14:39:56 on dist-test-slave-twwt"
server_key: "e097eaedc13b41b4d7783859d0323480"
server_key_iv: "515116b58bdbc37ba969de35ab26c65c"
server_key_version: "encryptionkey@0"
I20260811 14:39:57.001322 5852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "7c16508723fb4777bf9b56a89e56bbf8"
format_stamp: "Formatted at 2026-08-11 14:39:56 on dist-test-slave-twwt"
server_key: "e097eaedc13b41b4d7783859d0323480"
server_key_iv: "515116b58bdbc37ba969de35ab26c65c"
server_key_version: "encryptionkey@0"
I20260811 14:39:57.002882 5852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 14:39:57.003774 5881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:57.003989 5852 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:39:57.004096 5852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "7c16508723fb4777bf9b56a89e56bbf8"
format_stamp: "Formatted at 2026-08-11 14:39:56 on dist-test-slave-twwt"
server_key: "e097eaedc13b41b4d7783859d0323480"
server_key_iv: "515116b58bdbc37ba969de35ab26c65c"
server_key_version: "encryptionkey@0"
I20260811 14:39:57.004158 5852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:57.017814 5852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:57.020313 5852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:57.020443 5852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:57.024753 5852 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:33643
I20260811 14:39:57.024816 5961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:33643 every 8 connection(s)
I20260811 14:39:57.025171 5852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:57.026314 5962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:57.028734 5962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8: Bootstrap starting.
I20260811 14:39:57.029363 5962 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:57.029667 5962 log.cc:824] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:57.030391 5962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8: No bootstrap required, opened a new log
I20260811 14:39:57.032058 5962 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c16508723fb4777bf9b56a89e56bbf8" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33643 } }
I20260811 14:39:57.032227 5962 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:57.032276 5962 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c16508723fb4777bf9b56a89e56bbf8, State: Initialized, Role: FOLLOWER
I20260811 14:39:57.032454 5962 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c16508723fb4777bf9b56a89e56bbf8" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33643 } }
I20260811 14:39:57.032542 5962 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:57.032588 5962 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:57.032641 5962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:57.033551 5962 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c16508723fb4777bf9b56a89e56bbf8" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33643 } }
I20260811 14:39:57.033681 5962 leader_election.cc:302] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c16508723fb4777bf9b56a89e56bbf8; no voters:
I20260811 14:39:57.033881 5962 leader_election.cc:288] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:57.033987 5966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:57.034183 5966 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8 [term 1 LEADER]: Becoming Leader. State: Replica: 7c16508723fb4777bf9b56a89e56bbf8, State: Running, Role: LEADER
I20260811 14:39:57.034273 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 5852
I20260811 14:39:57.034281 5966 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c16508723fb4777bf9b56a89e56bbf8" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33643 } }
I20260811 14:39:57.034404 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:57.034647 2461 external_mini_cluster.cc:1428] Setting key cabdc0c7eb116b9efd521273fa181eaa
I20260811 14:39:57.034680 5967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c16508723fb4777bf9b56a89e56bbf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c16508723fb4777bf9b56a89e56bbf8" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33643 } } }
I20260811 14:39:57.034693 5970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c16508723fb4777bf9b56a89e56bbf8. Latest consensus state: current_term: 1 leader_uuid: "7c16508723fb4777bf9b56a89e56bbf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c16508723fb4777bf9b56a89e56bbf8" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33643 } } }
I20260811 14:39:57.034790 5967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:57.034796 5970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8 [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:57.034929 5962 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:57.035063 5973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:57.035629 5973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:57.037284 5973 catalog_manager.cc:1347] Generated new cluster ID: c6989333aedc4879b41e145a2a0525c4
I20260811 14:39:57.037340 5973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:39:57.040534 5978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:57.035193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55501 (local address 127.2.103.126:33643)
0811 14:39:57.035526 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:57.035537 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:57.035571 (+ 34us) server_negotiation.cc:407] Connection header received
0811 14:39:57.035696 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:57.035704 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:57.035764 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:57.036166 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:57.036278 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:57.037641 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:57.038462 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:57.038977 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:57.039109 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:57.039870 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:57.039873 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:57.039874 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:57.040062 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:57.040093 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:57.040105 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:57.040201 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:57.040286 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 14:39:57.040317 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":74,"threads_started":1}
I20260811 14:39:57.041424 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:39:57.043800 5973 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:57.044463 5973 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:57.046957 5978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:57.043145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41859 (local address 127.2.103.126:33643)
0811 14:39:57.043202 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:57.043204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:57.043455 (+ 251us) server_negotiation.cc:407] Connection header received
0811 14:39:57.043541 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:57.043542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:57.043554 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:57.043604 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:57.043783 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:57.044503 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:57.045528 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:39:57.045850 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:39:57.045887 (+ 37us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:39:57.046691 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:57.046692 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:57.046693 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:57.046697 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:57.046711 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:57.046715 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:57.046754 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:57.046873 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 14:39:57.046886 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 14:39:57.049541 5973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c16508723fb4777bf9b56a89e56bbf8: Generated new TSK 0
I20260811 14:39:57.049726 5973 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:39:57.051939 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 5852
W20260811 14:39:57.063453 5829 connection.cc:535] client connection to 127.2.103.126:33643 recv error: Network error: failed to read from TLS socket (remote: 127.2.103.126:33643): Connection reset by peer (error 104)
2026-08-11T14:39:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1572 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:39:57 dist-test-slave-twwt krb5kdc[6011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:39:57 dist-test-slave-twwt krb5kdc[6011](info): set up 2 sockets
Aug 11 14:39:57 dist-test-slave-twwt krb5kdc[6011](info): commencing operation
krb5kdc: starting...
W20260811 14:39:58.332563 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:39:58 dist-test-slave-twwt krb5kdc[6011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:39:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:39:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:39:58.466296 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:39643
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:38495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:39643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:39:58.552762 6031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:39:58.553000 6031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:39:58.553025 6031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:39:58.554335 6031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:39:58.554387 6031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:39:58.554401 6031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:39:58.554415 6031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:39:58.554466 6031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:39:58.556012 6031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:38495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:39643
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:39643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:39:58.556291 6031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:39:58.556537 6031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:39:58.561543 6043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:58.561550 6040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:39:58.561550 6050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:39:58.561954 6031 server_base.cc:1034] running on GCE node
I20260811 14:39:58.562212 6031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:39:58.562513 6031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:39:58.563674 6031 hybrid_clock.cc:648] HybridClock initialized: now 1786459198563661 us; error 49 us; skew 500 ppm
Aug 11 14:39:58 dist-test-slave-twwt krb5kdc[6011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459198, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:39:58.566392 6031 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:39:58.566958 6031 webserver.cc:460] Webserver started at http://127.2.103.126:40459/ using document root <none> and password file <none>
I20260811 14:39:58.567195 6031 fs_manager.cc:359] Metadata directory not provided
I20260811 14:39:58.567265 6031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:39:58.567425 6031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:39:58.568574 6031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "3b0a5ad8553e4de6a72fe68f627aedff"
format_stamp: "Formatted at 2026-08-11 14:39:58 on dist-test-slave-twwt"
server_key: "ccc8d01196f3b95a74f20d389e4c5ba5"
server_key_iv: "1aaec898ece502ca15aecd7225bf3e82"
server_key_version: "encryptionkey@0"
I20260811 14:39:58.568971 6031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "3b0a5ad8553e4de6a72fe68f627aedff"
format_stamp: "Formatted at 2026-08-11 14:39:58 on dist-test-slave-twwt"
server_key: "ccc8d01196f3b95a74f20d389e4c5ba5"
server_key_iv: "1aaec898ece502ca15aecd7225bf3e82"
server_key_version: "encryptionkey@0"
I20260811 14:39:58.570447 6031 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:39:58.571368 6063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:39:58.571605 6031 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:39:58.571728 6031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "3b0a5ad8553e4de6a72fe68f627aedff"
format_stamp: "Formatted at 2026-08-11 14:39:58 on dist-test-slave-twwt"
server_key: "ccc8d01196f3b95a74f20d389e4c5ba5"
server_key_iv: "1aaec898ece502ca15aecd7225bf3e82"
server_key_version: "encryptionkey@0"
I20260811 14:39:58.571851 6031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:39:58.575356 6031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:39:58.577883 6031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:39:58.578051 6031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:39:58.582444 6031 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:39643
I20260811 14:39:58.582496 6136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:39643 every 8 connection(s)
I20260811 14:39:58.582901 6031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:39:58.583932 6137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:39:58.586272 6137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff: Bootstrap starting.
I20260811 14:39:58.586936 6137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff: Neither blocks nor log segments found. Creating new log.
I20260811 14:39:58.587177 6137 log.cc:824] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff: Log is configured to *not* fsync() on all Append() calls
I20260811 14:39:58.587884 6137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff: No bootstrap required, opened a new log
I20260811 14:39:58.589475 6137 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0a5ad8553e4de6a72fe68f627aedff" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 39643 } }
I20260811 14:39:58.589620 6137 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:39:58.589643 6137 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b0a5ad8553e4de6a72fe68f627aedff, State: Initialized, Role: FOLLOWER
I20260811 14:39:58.589789 6137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0a5ad8553e4de6a72fe68f627aedff" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 39643 } }
I20260811 14:39:58.589877 6137 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:39:58.589902 6137 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:39:58.589933 6137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:39:58.590834 6137 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0a5ad8553e4de6a72fe68f627aedff" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 39643 } }
I20260811 14:39:58.590930 6137 leader_election.cc:302] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b0a5ad8553e4de6a72fe68f627aedff; no voters:
I20260811 14:39:58.591064 6137 leader_election.cc:288] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:39:58.591142 6141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:39:58.591336 6141 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff [term 1 LEADER]: Becoming Leader. State: Replica: 3b0a5ad8553e4de6a72fe68f627aedff, State: Running, Role: LEADER
I20260811 14:39:58.591387 6137 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:39:58.591439 6141 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0a5ad8553e4de6a72fe68f627aedff" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 39643 } }
I20260811 14:39:58.591871 6143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b0a5ad8553e4de6a72fe68f627aedff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0a5ad8553e4de6a72fe68f627aedff" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 39643 } } }
I20260811 14:39:58.591871 6144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b0a5ad8553e4de6a72fe68f627aedff. Latest consensus state: current_term: 1 leader_uuid: "3b0a5ad8553e4de6a72fe68f627aedff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0a5ad8553e4de6a72fe68f627aedff" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 39643 } } }
I20260811 14:39:58.592049 6143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:58.592056 6144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff [sys.catalog]: This master's current role is: LEADER
I20260811 14:39:58.592386 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 6031
I20260811 14:39:58.592526 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:39:58.592797 2461 external_mini_cluster.cc:1428] Setting key e6e2fa3bbcd993705ed82712b466718f
W20260811 14:39:58.593524 6164 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:39:58.593591 6164 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:39:58.593658 6155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:39:58.593851 6155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:39:58.596082 6169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:58.593387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42907 (local address 127.2.103.126:39643)
0811 14:39:58.593574 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:58.593584 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:58.593617 (+ 33us) server_negotiation.cc:407] Connection header received
0811 14:39:58.594597 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:58.594630 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:58.594673 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:58.594832 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:58.595580 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:58.595584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:58.595584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:58.595589 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:58.595640 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:58.595652 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:58.595714 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:58.595839 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 14:39:58.595885 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260811 14:39:58.596400 6155 catalog_manager.cc:1347] Generated new cluster ID: d6066cfa4f1a40c59150517e46e1ff73
I20260811 14:39:58.596460 6155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:39:58.599534 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:39:58.602228 6169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:39:58.601002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53549 (local address 127.2.103.126:39643)
0811 14:39:58.601046 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 14:39:58.601048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:39:58.601196 (+ 148us) server_negotiation.cc:407] Connection header received
0811 14:39:58.601275 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:39:58.601275 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:39:58.601281 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:39:58.601344 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:39:58.601988 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:39:58.601990 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:39:58.601990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:39:58.601994 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:39:58.602010 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:39:58.602014 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:39:58.602068 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 14:39:58.602125 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 14:39:58.602139 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:39:58.615329 6155 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:39:58.615535 6155 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:39:58.619727 6155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b0a5ad8553e4de6a72fe68f627aedff: Generated new TSK 0
I20260811 14:39:58.619971 6155 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:39:58.623574 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 6031
2026-08-11T14:39:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1574 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:39:58 dist-test-slave-twwt krb5kdc[6193](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:39:58 dist-test-slave-twwt krb5kdc[6193](info): set up 2 sockets
Aug 11 14:39:58 dist-test-slave-twwt krb5kdc[6193](info): commencing operation
krb5kdc: starting...
W20260811 14:39:59.895481 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:39:59 dist-test-slave-twwt krb5kdc[6193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:40:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:40:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:40:00.039935 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:43721
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:35273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:43721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:40:00.130962 6208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:40:00.131251 6208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:40:00.131291 6208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:40:00.133325 6208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:40:00.133423 6208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:40:00.133466 6208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:40:00.133514 6208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:40:00.133545 6208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:40:00.135527 6208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:35273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:43721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:43721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:40:00.135912 6208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:40:00.136199 6208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:40:00.141409 6215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:40:00.141409 6214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:40:00.141412 6218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:40:00.141659 6208 server_base.cc:1034] running on GCE node
I20260811 14:40:00.141968 6208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:40:00.142257 6208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:40:00.143401 6208 hybrid_clock.cc:648] HybridClock initialized: now 1786459200143376 us; error 45 us; skew 500 ppm
Aug 11 14:40:00 dist-test-slave-twwt krb5kdc[6193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459200, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:40:00.146181 6208 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:40:00.146785 6208 webserver.cc:460] Webserver started at http://127.2.103.126:38873/ using document root <none> and password file <none>
I20260811 14:40:00.147058 6208 fs_manager.cc:359] Metadata directory not provided
I20260811 14:40:00.147106 6208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:40:00.147286 6208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:40:00.148448 6208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "bc535b8118ea4b4697ef0625289e5639"
format_stamp: "Formatted at 2026-08-11 14:40:00 on dist-test-slave-twwt"
server_key: "aa9cf0ee10d526ed9cd8bc4330d37bb1"
server_key_iv: "c8d12b532c0574b15441f5db03e02b00"
server_key_version: "encryptionkey@0"
I20260811 14:40:00.148835 6208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "bc535b8118ea4b4697ef0625289e5639"
format_stamp: "Formatted at 2026-08-11 14:40:00 on dist-test-slave-twwt"
server_key: "aa9cf0ee10d526ed9cd8bc4330d37bb1"
server_key_iv: "c8d12b532c0574b15441f5db03e02b00"
server_key_version: "encryptionkey@0"
I20260811 14:40:00.150314 6208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:40:00.151283 6229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:40:00.151583 6208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:40:00.151676 6208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "bc535b8118ea4b4697ef0625289e5639"
format_stamp: "Formatted at 2026-08-11 14:40:00 on dist-test-slave-twwt"
server_key: "aa9cf0ee10d526ed9cd8bc4330d37bb1"
server_key_iv: "c8d12b532c0574b15441f5db03e02b00"
server_key_version: "encryptionkey@0"
I20260811 14:40:00.151777 6208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:40:00.161666 6208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:40:00.164263 6208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:40:00.164419 6208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:40:00.168964 6208 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:43721
I20260811 14:40:00.169003 6313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:43721 every 8 connection(s)
I20260811 14:40:00.169390 6208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:40:00.170351 6314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:40:00.172887 6314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639: Bootstrap starting.
I20260811 14:40:00.173556 6314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639: Neither blocks nor log segments found. Creating new log.
I20260811 14:40:00.173877 6314 log.cc:824] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639: Log is configured to *not* fsync() on all Append() calls
I20260811 14:40:00.174795 6314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639: No bootstrap required, opened a new log
I20260811 14:40:00.176558 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 6208
I20260811 14:40:00.176483 6314 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc535b8118ea4b4697ef0625289e5639" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 43721 } }
I20260811 14:40:00.176652 6314 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:40:00.176672 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:40:00.176700 6314 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc535b8118ea4b4697ef0625289e5639, State: Initialized, Role: FOLLOWER
I20260811 14:40:00.176860 6314 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc535b8118ea4b4697ef0625289e5639" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 43721 } }
I20260811 14:40:00.176929 2461 external_mini_cluster.cc:1428] Setting key 80b6dac43aff0cc7b6f296691af9519b
I20260811 14:40:00.176954 6314 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:40:00.176997 6314 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:40:00.177052 6314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:40:00.178146 6314 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc535b8118ea4b4697ef0625289e5639" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 43721 } }
I20260811 14:40:00.178297 6314 leader_election.cc:302] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc535b8118ea4b4697ef0625289e5639; no voters:
I20260811 14:40:00.178490 6314 leader_election.cc:288] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:40:00.178535 6319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:40:00.178771 6319 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639 [term 1 LEADER]: Becoming Leader. State: Replica: bc535b8118ea4b4697ef0625289e5639, State: Running, Role: LEADER
I20260811 14:40:00.178828 6314 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:40:00.178880 6319 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc535b8118ea4b4697ef0625289e5639" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 43721 } }
I20260811 14:40:00.179288 6319 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc535b8118ea4b4697ef0625289e5639. Latest consensus state: current_term: 1 leader_uuid: "bc535b8118ea4b4697ef0625289e5639" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc535b8118ea4b4697ef0625289e5639" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 43721 } } }
I20260811 14:40:00.179378 6319 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639 [sys.catalog]: This master's current role is: LEADER
I20260811 14:40:00.179302 6321 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc535b8118ea4b4697ef0625289e5639" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc535b8118ea4b4697ef0625289e5639" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 43721 } } }
I20260811 14:40:00.179579 6328 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:40:00.179597 6321 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639 [sys.catalog]: This master's current role is: LEADER
I20260811 14:40:00.180099 6328 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:40:00.182973 6328 catalog_manager.cc:1347] Generated new cluster ID: 45074206b8b74d08ba05170dbc5a12ab
I20260811 14:40:00.183041 6328 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:40:00 dist-test-slave-twwt krb5kdc[6193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:40:00.190173 6318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:00.177548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56257 (local address 127.2.103.126:43721)
0811 14:40:00.177756 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:00.177768 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:00.177810 (+ 42us) server_negotiation.cc:407] Connection header received
0811 14:40:00.177937 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:00.177943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:00.177990 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:00.178129 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:00.179073 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:00.180585 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:00.181497 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:00.182121 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:00.182259 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:00.185396 (+ 3137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:00.185402 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:00.185403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:00.185421 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:00.187645 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:00.188153 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:00.188158 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:00.188158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:00.188312 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:00.188541 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:00.188542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:00.188543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:00.188928 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:00.189125 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:00.189762 (+ 637us) server_negotiation.cc:299] Negotiation successful
0811 14:40:00.189929 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":111,"threads_started":1}
I20260811 14:40:00.191231 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:40:00.191907 6328 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:40:00.192652 6328 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:40:00.200686 6318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:00.192873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54169 (local address 127.2.103.126:43721)
0811 14:40:00.192913 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:00.192915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:00.193105 (+ 190us) server_negotiation.cc:407] Connection header received
0811 14:40:00.193144 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:00.193145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:00.193157 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:00.193222 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:00.193874 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:00.194690 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:00.195739 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:00.196093 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:00.196168 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:00.197720 (+ 1552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:00.197721 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:00.197722 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:00.197731 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:00.199472 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:00.199881 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:00.199882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:00.199883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:00.199949 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:00.200149 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:00.200150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:00.200151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:00.200289 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:00.200395 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:00.200496 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 14:40:00.200582 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 14:40:00.203002 6328 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc535b8118ea4b4697ef0625289e5639: Generated new TSK 0
I20260811 14:40:00.203213 6328 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:40:00.206974 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 6208
2026-08-11T14:40:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1586 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:40:00 dist-test-slave-twwt krb5kdc[6364](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:40:00 dist-test-slave-twwt krb5kdc[6364](info): set up 2 sockets
Aug 11 14:40:00 dist-test-slave-twwt krb5kdc[6364](info): commencing operation
krb5kdc: starting...
W20260811 14:40:01.484459 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:40:01 dist-test-slave-twwt krb5kdc[6364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:40:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:40:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:40:01.622355 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:33241
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:42615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:33241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:40:01.711858 6382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:40:01.712152 6382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:40:01.712199 6382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:40:01.713482 6382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:40:01.713569 6382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:40:01.713594 6382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:40:01.713615 6382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:40:01.713645 6382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:40:01.715324 6382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:42615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:33241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:33241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:40:01.715647 6382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:40:01.715903 6382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:40:01.721240 6391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:40:01.721256 6394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:40:01.721302 6382 server_base.cc:1034] running on GCE node
W20260811 14:40:01.721262 6390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:40:01.721698 6382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:40:01.722059 6382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:40:01.723242 6382 hybrid_clock.cc:648] HybridClock initialized: now 1786459201723217 us; error 41 us; skew 500 ppm
Aug 11 14:40:01 dist-test-slave-twwt krb5kdc[6364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459201, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:40:01.726074 6382 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:40:01.726676 6382 webserver.cc:460] Webserver started at http://127.2.103.126:44529/ using document root <none> and password file <none>
I20260811 14:40:01.726943 6382 fs_manager.cc:359] Metadata directory not provided
I20260811 14:40:01.726994 6382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:40:01.727133 6382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:40:01.728242 6382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "0a1a9c465eff431a8c19491c812f3fa3"
format_stamp: "Formatted at 2026-08-11 14:40:01 on dist-test-slave-twwt"
server_key: "2f6313edce5b6182657032f8a5c9a535"
server_key_iv: "568d1b5e913639a5d0540bdaff3e6c90"
server_key_version: "encryptionkey@0"
I20260811 14:40:01.728660 6382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "0a1a9c465eff431a8c19491c812f3fa3"
format_stamp: "Formatted at 2026-08-11 14:40:01 on dist-test-slave-twwt"
server_key: "2f6313edce5b6182657032f8a5c9a535"
server_key_iv: "568d1b5e913639a5d0540bdaff3e6c90"
server_key_version: "encryptionkey@0"
I20260811 14:40:01.730144 6382 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:40:01.731020 6407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:40:01.731261 6382 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:40:01.731376 6382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "0a1a9c465eff431a8c19491c812f3fa3"
format_stamp: "Formatted at 2026-08-11 14:40:01 on dist-test-slave-twwt"
server_key: "2f6313edce5b6182657032f8a5c9a535"
server_key_iv: "568d1b5e913639a5d0540bdaff3e6c90"
server_key_version: "encryptionkey@0"
I20260811 14:40:01.731452 6382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:40:01.769299 6382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:40:01.772030 6382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:40:01.772157 6382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:40:01.776705 6382 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:33241
I20260811 14:40:01.776743 6493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:33241 every 8 connection(s)
I20260811 14:40:01.777170 6382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:40:01.778100 6494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:40:01.779670 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 6382
I20260811 14:40:01.779845 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:40:01.780047 2461 external_mini_cluster.cc:1428] Setting key 054939c7e4714ba84f5a18d28fe38f1f
I20260811 14:40:01.781036 6494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3: Bootstrap starting.
I20260811 14:40:01.781810 6494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3: Neither blocks nor log segments found. Creating new log.
I20260811 14:40:01.782155 6494 log.cc:824] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3: Log is configured to *not* fsync() on all Append() calls
I20260811 14:40:01.782999 6494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3: No bootstrap required, opened a new log
I20260811 14:40:01.785308 6494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a1a9c465eff431a8c19491c812f3fa3" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33241 } }
I20260811 14:40:01.785498 6494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:40:01.785542 6494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1a9c465eff431a8c19491c812f3fa3, State: Initialized, Role: FOLLOWER
I20260811 14:40:01.785689 6494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a1a9c465eff431a8c19491c812f3fa3" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33241 } }
I20260811 14:40:01.785791 6494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:40:01.785825 6494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:40:01.785869 6494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:40:01.786504 6503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:01.780647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38187 (local address 127.2.103.126:33241)
0811 14:40:01.780875 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:01.780886 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:01.780920 (+ 34us) server_negotiation.cc:407] Connection header received
0811 14:40:01.781490 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:01.781502 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:01.781568 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:01.781736 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:01.782003 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:01.783381 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:01.784207 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:01.784924 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:01.785055 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:01.785772 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:01.785777 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:01.785777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:40:01.786012 (+ 235us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:01.786054 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:40:01.786069 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:01.786138 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:01.786209 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 14:40:01.786279 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":94,"threads_started":1}
I20260811 14:40:01.786680 6494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a1a9c465eff431a8c19491c812f3fa3" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33241 } }
I20260811 14:40:01.786820 6494 leader_election.cc:302] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a1a9c465eff431a8c19491c812f3fa3; no voters:
I20260811 14:40:01.786994 6494 leader_election.cc:288] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:40:01.787225 6511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:40:01.787425 6511 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3 [term 1 LEADER]: Becoming Leader. State: Replica: 0a1a9c465eff431a8c19491c812f3fa3, State: Running, Role: LEADER
I20260811 14:40:01.787525 6494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:40:01.787580 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:40:01.787564 6511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a1a9c465eff431a8c19491c812f3fa3" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33241 } }
I20260811 14:40:01.788857 6516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a1a9c465eff431a8c19491c812f3fa3. Latest consensus state: current_term: 1 leader_uuid: "0a1a9c465eff431a8c19491c812f3fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a1a9c465eff431a8c19491c812f3fa3" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33241 } } }
I20260811 14:40:01.788952 6516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3 [sys.catalog]: This master's current role is: LEADER
I20260811 14:40:01.789206 6544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:40:01.789832 6544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:40:01.790545 6514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a1a9c465eff431a8c19491c812f3fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a1a9c465eff431a8c19491c812f3fa3" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 33241 } } }
I20260811 14:40:01.790730 6514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3 [sys.catalog]: This master's current role is: LEADER
I20260811 14:40:01.791553 6544 catalog_manager.cc:1347] Generated new cluster ID: 2d684d5be47340599718d851d14f1cb9
I20260811 14:40:01.791611 6544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:40:01.792881 6503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:01.789193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36537 (local address 127.2.103.126:33241)
0811 14:40:01.789232 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:01.789234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:01.789432 (+ 198us) server_negotiation.cc:407] Connection header received
0811 14:40:01.789541 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:01.789542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:01.789553 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:01.789614 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:01.789781 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:01.790323 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:01.791507 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:01.791808 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:01.791850 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:01.792601 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:01.792602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:01.792602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:40:01.792607 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:01.792621 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:40:01.792626 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:01.792680 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:01.792753 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 14:40:01.792777 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 14:40:01.801923 6544 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:40:01.802745 6544 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:40:01.809427 6544 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a1a9c465eff431a8c19491c812f3fa3: Generated new TSK 0
I20260811 14:40:01.809649 6544 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:40:01.817050 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 6382
2026-08-11T14:40:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1611 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:40:01 dist-test-slave-twwt krb5kdc[6554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:40:01 dist-test-slave-twwt krb5kdc[6554](info): set up 2 sockets
Aug 11 14:40:01 dist-test-slave-twwt krb5kdc[6554](info): commencing operation
krb5kdc: starting...
W20260811 14:40:03.130540 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:40:03 dist-test-slave-twwt krb5kdc[6554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:40:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:40:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:40:03.266454 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:36487
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:37455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:36487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:40:03.355115 6575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:40:03.355322 6575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:40:03.355345 6575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:40:03.356446 6575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:40:03.356484 6575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:40:03.356498 6575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:40:03.356511 6575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:40:03.356523 6575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:40:03.357952 6575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:37455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:36487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:36487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:40:03.358146 6575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:40:03.358319 6575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:40:03.363790 6582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:40:03.363822 6584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:40:03.363840 6586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:40:03.364104 6575 server_base.cc:1034] running on GCE node
I20260811 14:40:03.364470 6575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:40:03.364723 6575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:40:03.365863 6575 hybrid_clock.cc:648] HybridClock initialized: now 1786459203365859 us; error 37 us; skew 500 ppm
Aug 11 14:40:03 dist-test-slave-twwt krb5kdc[6554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459203, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:40:03.368575 6575 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:40:03.368978 6575 webserver.cc:460] Webserver started at http://127.2.103.126:43771/ using document root <none> and password file <none>
I20260811 14:40:03.369212 6575 fs_manager.cc:359] Metadata directory not provided
I20260811 14:40:03.369283 6575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:40:03.369441 6575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:40:03.370545 6575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "1119dd76aadf4709a2d67f1e683ab2ea"
format_stamp: "Formatted at 2026-08-11 14:40:03 on dist-test-slave-twwt"
server_key: "94ef121123c54ccc3ceb74bd0bbe6c05"
server_key_iv: "b9ef7328081d82a6fc38a07e908f8193"
server_key_version: "encryptionkey@0"
I20260811 14:40:03.370983 6575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "1119dd76aadf4709a2d67f1e683ab2ea"
format_stamp: "Formatted at 2026-08-11 14:40:03 on dist-test-slave-twwt"
server_key: "94ef121123c54ccc3ceb74bd0bbe6c05"
server_key_iv: "b9ef7328081d82a6fc38a07e908f8193"
server_key_version: "encryptionkey@0"
I20260811 14:40:03.372537 6575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:40:03.373418 6595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:40:03.373636 6575 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:40:03.373750 6575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "1119dd76aadf4709a2d67f1e683ab2ea"
format_stamp: "Formatted at 2026-08-11 14:40:03 on dist-test-slave-twwt"
server_key: "94ef121123c54ccc3ceb74bd0bbe6c05"
server_key_iv: "b9ef7328081d82a6fc38a07e908f8193"
server_key_version: "encryptionkey@0"
I20260811 14:40:03.373864 6575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:40:03.377291 6575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:40:03.379415 6575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:40:03.379576 6575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:40:03.383863 6575 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:36487
I20260811 14:40:03.383901 6674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:36487 every 8 connection(s)
I20260811 14:40:03.384258 6575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:40:03.385372 6675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:40:03.387838 6675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea: Bootstrap starting.
I20260811 14:40:03.388440 6675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea: Neither blocks nor log segments found. Creating new log.
I20260811 14:40:03.388757 6675 log.cc:824] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea: Log is configured to *not* fsync() on all Append() calls
I20260811 14:40:03.389513 6675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea: No bootstrap required, opened a new log
I20260811 14:40:03.391284 6675 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1119dd76aadf4709a2d67f1e683ab2ea" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36487 } }
I20260811 14:40:03.391429 6675 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:40:03.391453 6675 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1119dd76aadf4709a2d67f1e683ab2ea, State: Initialized, Role: FOLLOWER
I20260811 14:40:03.391638 6675 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1119dd76aadf4709a2d67f1e683ab2ea" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36487 } }
I20260811 14:40:03.391736 6675 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:40:03.391796 6675 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:40:03.391844 6675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:40:03.392716 6675 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1119dd76aadf4709a2d67f1e683ab2ea" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36487 } }
I20260811 14:40:03.392717 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 6575
I20260811 14:40:03.392817 6675 leader_election.cc:302] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1119dd76aadf4709a2d67f1e683ab2ea; no voters:
I20260811 14:40:03.392870 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:40:03.393069 2461 external_mini_cluster.cc:1428] Setting key bec5383b09ef66e616c15e972194462f
I20260811 14:40:03.393054 6675 leader_election.cc:288] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:40:03.393082 6681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:40:03.393216 6681 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea [term 1 LEADER]: Becoming Leader. State: Replica: 1119dd76aadf4709a2d67f1e683ab2ea, State: Running, Role: LEADER
I20260811 14:40:03.393359 6675 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:40:03.393386 6681 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1119dd76aadf4709a2d67f1e683ab2ea" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36487 } }
I20260811 14:40:03.393827 6682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1119dd76aadf4709a2d67f1e683ab2ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1119dd76aadf4709a2d67f1e683ab2ea" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36487 } } }
I20260811 14:40:03.393919 6682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea [sys.catalog]: This master's current role is: LEADER
I20260811 14:40:03.394146 6683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1119dd76aadf4709a2d67f1e683ab2ea. Latest consensus state: current_term: 1 leader_uuid: "1119dd76aadf4709a2d67f1e683ab2ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1119dd76aadf4709a2d67f1e683ab2ea" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 36487 } } }
I20260811 14:40:03.394287 6683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea [sys.catalog]: This master's current role is: LEADER
I20260811 14:40:03.394456 6692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:40:03.395285 6692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:40:03.395954 6686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:03.393721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36815 (local address 127.2.103.126:36487)
0811 14:40:03.393914 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:03.393923 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:03.393958 (+ 35us) server_negotiation.cc:407] Connection header received
0811 14:40:03.394090 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:03.394099 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:03.394136 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:03.394592 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:03.395266 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:03.395272 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:03.395272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:40:03.395277 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:03.395327 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:40:03.395343 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:03.395681 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:03.395715 (+ 34us) server_negotiation.cc:299] Negotiation successful
0811 14:40:03.395769 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":81,"threads_started":1}
I20260811 14:40:03.396981 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:40:03.397156 6692 catalog_manager.cc:1347] Generated new cluster ID: 3ff69b9925764ff883d32de921090dca
I20260811 14:40:03.397199 6692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:40:03.401279 6692 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:40:03.401474 6692 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:40:03.401968 6686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:03.400860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53201 (local address 127.2.103.126:36487)
0811 14:40:03.400894 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:03.400896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:03.400905 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:40:03.400937 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:03.400938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:03.400944 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:03.401014 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:03.401720 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:03.401721 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:03.401722 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:40:03.401726 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:03.401748 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:40:03.401753 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:03.401810 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:03.401891 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 14:40:03.401904 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 14:40:03.410218 6692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1119dd76aadf4709a2d67f1e683ab2ea: Generated new TSK 0
I20260811 14:40:03.410470 6692 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:40:03.418324 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 6575
2026-08-11T14:40:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1596 ms)
[----------] 18 tests from AuthTokenIssuingTest (28914 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:40:03 dist-test-slave-twwt krb5kdc[6726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:40:03 dist-test-slave-twwt krb5kdc[6726](info): set up 2 sockets
Aug 11 14:40:03 dist-test-slave-twwt krb5kdc[6726](info): commencing operation
krb5kdc: starting...
W20260811 14:40:04.702524 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:40:04 dist-test-slave-twwt krb5kdc[6726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:40:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:40:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:40:04.838246 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:35185
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:33991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:35185,127.2.103.125:37135,127.2.103.124:43553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:40:04.924185 6745 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 14:40:04.924465 6745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:40:04.924501 6745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:40:04.924544 6745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:40:04.925726 6745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:40:04.925804 6745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:40:04.925829 6745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:40:04.925850 6745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:40:04.925880 6745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:40:04.927441 6745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:33991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:35185,127.2.103.125:37135,127.2.103.124:43553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:35185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:40:04.927745 6745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:40:04.927997 6745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:40:04.933171 6756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:40:04.933200 6755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:40:04.933228 6759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:40:04.933311 6745 server_base.cc:1034] running on GCE node
I20260811 14:40:04.933656 6745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:40:04.934008 6745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:40:04.935177 6745 hybrid_clock.cc:648] HybridClock initialized: now 1786459204935158 us; error 36 us; skew 500 ppm
Aug 11 14:40:04 dist-test-slave-twwt krb5kdc[6726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459204, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:40:04.942946 6745 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:40:04.943635 6745 webserver.cc:460] Webserver started at http://127.2.103.126:32827/ using document root <none> and password file <none>
I20260811 14:40:04.943944 6745 fs_manager.cc:359] Metadata directory not provided
I20260811 14:40:04.944020 6745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:40:04.944187 6745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:40:04.945925 6745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "ef32f1c0f6c248209c898fa339fbc57c"
format_stamp: "Formatted at 2026-08-11 14:40:04 on dist-test-slave-twwt"
server_key: "9e91c0d1d5dedc137b5eca2b9bc92716"
server_key_iv: "7451d7cb28bf109f25dd216b6d4d9a66"
server_key_version: "encryptionkey@0"
I20260811 14:40:04.946414 6745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "ef32f1c0f6c248209c898fa339fbc57c"
format_stamp: "Formatted at 2026-08-11 14:40:04 on dist-test-slave-twwt"
server_key: "9e91c0d1d5dedc137b5eca2b9bc92716"
server_key_iv: "7451d7cb28bf109f25dd216b6d4d9a66"
server_key_version: "encryptionkey@0"
I20260811 14:40:04.948493 6745 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260811 14:40:04.950903 6770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:40:04.951056 6745 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 14:40:04.951138 6745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "ef32f1c0f6c248209c898fa339fbc57c"
format_stamp: "Formatted at 2026-08-11 14:40:04 on dist-test-slave-twwt"
server_key: "9e91c0d1d5dedc137b5eca2b9bc92716"
server_key_iv: "7451d7cb28bf109f25dd216b6d4d9a66"
server_key_version: "encryptionkey@0"
I20260811 14:40:04.951205 6745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:40:04.994135 6745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:40:04.996382 6745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:40:04.996502 6745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:40:05.000685 6745 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:35185
I20260811 14:40:05.000749 6860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:35185 every 8 connection(s)
I20260811 14:40:05.001124 6745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:40:05.002238 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:40:05.003993 6862 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:40:05.005398 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 6745
I20260811 14:40:05.005496 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:40:05.005702 2461 external_mini_cluster.cc:1428] Setting key b4bbeafbfff4f6395174e001b1e30d3c
Aug 11 14:40:05 dist-test-slave-twwt krb5kdc[6726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459204, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.103.125@KRBTEST.COM; defaulting to no policy
I20260811 14:40:05.015848 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.004732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:35185 (local address 127.0.0.1:44514)
0811 14:40:05.004995 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.005043 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 14:40:05.005669 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:40:05.005978 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:40:05.005981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:40:05.006280 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:40:05.006451 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:05.006455 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.007671 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.007673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:05.008377 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:05.008380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.009097 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.009098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:05.009153 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.009711 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:40:05.009719 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:40:05.011514 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:40:05.013950 (+ 2436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:05.013957 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:05.013959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:05.014168 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:05.014480 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:05.014481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:05.014481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:05.014580 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:05.015223 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:40:05.015230 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:40:05.015296 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 14:40:05.015380 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 14:40:05.015524 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":95,"threads_started":1}
I20260811 14:40:05.015856 6868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.004801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44514 (local address 127.2.103.126:35185)
0811 14:40:05.005005 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.005015 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.005148 (+ 133us) server_negotiation.cc:407] Connection header received
0811 14:40:05.005844 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.005846 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.005864 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.005971 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.006569 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.007552 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.008472 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.008986 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.009099 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.011616 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:05.011626 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:05.011627 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:05.011635 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:05.013845 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.014262 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.014267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.014267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.014374 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.014793 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.014795 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.014796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.015137 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:05.015236 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:05.015443 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 14:40:05.015618 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":154,"thread_start_us":78,"threads_started":1}
I20260811 14:40:05.016836 6862 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:40:05.017237 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.017113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:37135 (local address 127.0.0.1:50378)
0811 14:40:05.017161 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.017216 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.125:37135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 14:40:05.017416 6775 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.103.125:37135: connect: Connection refused (error 111)
I20260811 14:40:05.017931 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.017810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:37135 (local address 127.0.0.1:50390)
0811 14:40:05.017879 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.017910 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.125:37135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 14:40:05.018182 6862 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.125:37135: Network error: Client connection negotiation failed: client connection to 127.2.103.125:37135: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.125.keytab.
Entry for principal kudu/127.2.103.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.125.keytab.
I20260811 14:40:05.038498 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.125:37135
--webserver_interface=127.2.103.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.125.keytab
--principal=kudu/127.2.103.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:33991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:35185,127.2.103.125:37135,127.2.103.124:43553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:40:05.067457 6862 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } attempt: 1
I20260811 14:40:05.068111 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.067928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:37135 (local address 127.0.0.1:50404)
0811 14:40:05.068006 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.068062 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.125:37135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 14:40:05.068581 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.068477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:37135 (local address 127.0.0.1:50418)
0811 14:40:05.068525 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.068556 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.125:37135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 14:40:05.068722 6862 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.125:37135: Network error: Client connection negotiation failed: client connection to 127.2.103.125:37135: connect: Connection refused (error 111)
W20260811 14:40:05.124660 6874 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 14:40:05.124907 6874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:40:05.124925 6874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:40:05.124948 6874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:40:05.126180 6874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:40:05.126235 6874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:40:05.126250 6874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:40:05.126264 6874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:40:05.126276 6874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:40:05.127902 6874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:33991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:35185,127.2.103.125:37135,127.2.103.124:43553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.125.keytab
--principal=kudu/127.2.103.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.125:37135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.103.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:40:05.128202 6874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:40:05.128470 6874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:40:05.133705 6884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:40:05.133710 6886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:40:05.133759 6874 server_base.cc:1034] running on GCE node
W20260811 14:40:05.133710 6883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:40:05.134238 6874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:40:05.134582 6874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:40:05.135838 6874 hybrid_clock.cc:648] HybridClock initialized: now 1786459205135824 us; error 34 us; skew 500 ppm
I20260811 14:40:05.136942 6862 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } attempt: 2
I20260811 14:40:05.137404 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.137262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:37135 (local address 127.0.0.1:50428)
0811 14:40:05.137329 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.137365 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.125:37135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 14:40:05.137735 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.137639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:37135 (local address 127.0.0.1:50436)
0811 14:40:05.137686 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.137711 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.125:37135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 14:40:05.137842 6862 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.125:37135: Network error: Client connection negotiation failed: client connection to 127.2.103.125:37135: connect: Connection refused (error 111)
Aug 11 14:40:05 dist-test-slave-twwt krb5kdc[6726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459205, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:40:05.138871 6874 init.cc:377] Logged in from keytab as kudu/127.2.103.125@KRBTEST.COM (short username kudu)
I20260811 14:40:05.139359 6874 webserver.cc:460] Webserver started at http://127.2.103.125:33711/ using document root <none> and password file <none>
I20260811 14:40:05.139599 6874 fs_manager.cc:359] Metadata directory not provided
I20260811 14:40:05.139647 6874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:40:05.139802 6874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:40:05.140933 6874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-1/data/instance:
uuid: "ceab8363121e41d7ab17360eb8397ec3"
format_stamp: "Formatted at 2026-08-11 14:40:05 on dist-test-slave-twwt"
server_key: "2c86c78c10d9e1aa4cfcac0ae2f6d915"
server_key_iv: "88a62cd420dcb90a3c1d6b3994d07ae3"
server_key_version: "encryptionkey@0"
I20260811 14:40:05.141321 6874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-1/wal/instance:
uuid: "ceab8363121e41d7ab17360eb8397ec3"
format_stamp: "Formatted at 2026-08-11 14:40:05 on dist-test-slave-twwt"
server_key: "2c86c78c10d9e1aa4cfcac0ae2f6d915"
server_key_iv: "88a62cd420dcb90a3c1d6b3994d07ae3"
server_key_version: "encryptionkey@0"
I20260811 14:40:05.142912 6874 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260811 14:40:05.143703 6897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:40:05.143896 6874 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:40:05.143991 6874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-1/wal
uuid: "ceab8363121e41d7ab17360eb8397ec3"
format_stamp: "Formatted at 2026-08-11 14:40:05 on dist-test-slave-twwt"
server_key: "2c86c78c10d9e1aa4cfcac0ae2f6d915"
server_key_iv: "88a62cd420dcb90a3c1d6b3994d07ae3"
server_key_version: "encryptionkey@0"
I20260811 14:40:05.144050 6874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:40:05.159897 6874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:40:05.162271 6874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:40:05.162405 6874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:40:05.166767 6874 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.125:37135
I20260811 14:40:05.166802 6987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.125:37135 every 8 connection(s)
I20260811 14:40:05.167217 6874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-1/data/info.pb
I20260811 14:40:05.168372 6988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:40:05.169981 6988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:40:05.174889 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 6874
I20260811 14:40:05.175009 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-1/wal/instance
I20260811 14:40:05.175207 2461 external_mini_cluster.cc:1428] Setting key 06aceda63af3cb8066d68620c8dcf33f
Aug 11 14:40:05 dist-test-slave-twwt krb5kdc[6726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459205, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.125@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:40:05.180655 6868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.170825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44524 (local address 127.2.103.126:35185)
0811 14:40:05.170877 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.170879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.171307 (+ 428us) server_negotiation.cc:407] Connection header received
0811 14:40:05.171685 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.171686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.171728 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.171813 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.172490 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.173036 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.174015 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.174222 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.174279 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.177154 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:05.177156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:05.177157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:05.177167 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:05.178979 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.179519 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.179519 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.179520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.179576 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.179879 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.179879 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.179879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.179997 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:05.180050 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:05.180441 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 14:40:05.180517 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 14:40:05.180951 6990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.170817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:35185 (local address 127.0.0.1:44524)
0811 14:40:05.170995 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.171035 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 14:40:05.171557 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:40:05.171899 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:40:05.171902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:40:05.172176 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:40:05.172394 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:05.172398 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.173124 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.173125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:05.173938 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:05.173939 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.174318 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.174319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:05.174435 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.175097 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:40:05.175106 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:40:05.176986 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:40:05.179079 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:05.179088 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:05.179091 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:05.179440 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:05.179641 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:05.179642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:05.179642 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:05.179800 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:05.180063 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:40:05.180070 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:40:05.180282 (+ 212us) client_negotiation.cc:769] Sending connection context
0811 14:40:05.180422 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 14:40:05.180630 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":93,"threads_started":1}
I20260811 14:40:05.181669 6988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.103.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.124@KRBTEST.COM" created.
Aug 11 14:40:05 dist-test-slave-twwt krb5kdc[6726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459205, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.125@KRBTEST.COM for kudu/127.2.103.125@KRBTEST.COM
I20260811 14:40:05.193729 6990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.182055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:37135 (local address 127.0.0.1:50450)
0811 14:40:05.182119 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.182132 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:40:05.182184 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:40:05.182540 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:40:05.182541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:40:05.182791 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:40:05.182850 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:05.182852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.184183 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.184184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:05.184725 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:05.184727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.185459 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.185461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:05.185522 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.186074 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:40:05.186079 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:40:05.189839 (+ 3760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:40:05.192420 (+ 2581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:05.192421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:05.192421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:05.192608 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:05.192919 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:05.192919 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:05.192920 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:05.192979 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:05.193327 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:40:05.193328 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:40:05.193374 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 14:40:05.193467 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 14:40:05.193607 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 14:40:05.193795 6995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.182147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50450 (local address 127.2.103.125:37135)
0811 14:40:05.182336 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.182343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.182371 (+ 28us) server_negotiation.cc:407] Connection header received
0811 14:40:05.182398 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.182400 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.182417 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.182508 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.182936 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.184070 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.184820 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.185367 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.185446 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.189988 (+ 4542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:05.190001 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:05.190001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:05.190009 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:05.192247 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.192773 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.192778 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.192778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.192838 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.193045 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.193046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.193046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.193222 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:05.193339 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:05.193507 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 14:40:05.193668 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":148,"thread_start_us":96,"threads_started":1}
I20260811 14:40:05.194831 6988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:40:05.195221 6990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.195097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:43553 (local address 127.0.0.1:42420)
0811 14:40:05.195149 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.195198 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:43553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 14:40:05.195379 6904 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.103.124:43553: connect: Connection refused (error 111)
I20260811 14:40:05.195880 6990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.195753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:43553 (local address 127.0.0.1:42426)
0811 14:40:05.195827 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.195854 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:43553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 14:40:05.195989 6988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.124:43553: Network error: Client connection negotiation failed: client connection to 127.2.103.124:43553: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.124.keytab.
Entry for principal kudu/127.2.103.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.124.keytab.
I20260811 14:40:05.206828 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.124:43553
--webserver_interface=127.2.103.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.124.keytab
--principal=kudu/127.2.103.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:33991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:35185,127.2.103.125:37135,127.2.103.124:43553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:40:05.229012 6862 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } attempt: 3
Aug 11 14:40:05 dist-test-slave-twwt krb5kdc[6726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459204, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for kudu/127.2.103.125@KRBTEST.COM
I20260811 14:40:05.238021 6995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.229543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50452 (local address 127.2.103.125:37135)
0811 14:40:05.229617 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.229620 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.229632 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:40:05.229710 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.229711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.229729 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.229833 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.230189 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.230755 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.231532 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.231766 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.231845 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.234059 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:05.234061 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:05.234061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:05.234072 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:05.236745 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.237164 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.237166 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.237166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.237235 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.237450 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.237451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.237452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.237578 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:05.237652 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:05.237827 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 14:40:05.237915 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 14:40:05.238021 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.229476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:37135 (local address 127.0.0.1:50452)
0811 14:40:05.229547 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.229558 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:40:05.229639 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:40:05.229841 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:40:05.229842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:40:05.230028 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:40:05.230099 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:05.230101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.230844 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.230846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:05.231388 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:05.231390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.231843 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.231844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:05.231935 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.232418 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:40:05.232428 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:40:05.233946 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:40:05.236880 (+ 2934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:05.236881 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:05.236882 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:05.237079 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:05.237310 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:05.237310 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:05.237311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:05.237383 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:05.237656 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:40:05.237658 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:40:05.237712 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 14:40:05.237798 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 14:40:05.237915 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 14:40:05.238936 6862 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:40:05.239342 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.239213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:43553 (local address 127.0.0.1:42434)
0811 14:40:05.239280 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.239316 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:43553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 14:40:05.239662 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.239606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:43553 (local address 127.0.0.1:42436)
0811 14:40:05.239628 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.239649 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:43553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260811 14:40:05.239765 6862 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.124:43553: Network error: Client connection negotiation failed: client connection to 127.2.103.124:43553: connect: Connection refused (error 111)
I20260811 14:40:05.245229 6988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } attempt: 1
I20260811 14:40:05.245692 6990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.245562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:43553 (local address 127.0.0.1:42444)
0811 14:40:05.245623 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.245661 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:43553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 14:40:05.246033 6990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.245933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:43553 (local address 127.0.0.1:42454)
0811 14:40:05.245980 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.246008 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:43553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 14:40:05.246126 6988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.124:43553: Network error: Client connection negotiation failed: client connection to 127.2.103.124:43553: connect: Connection refused (error 111)
I20260811 14:40:05.270953 6862 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } attempt: 1
I20260811 14:40:05.271551 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.271393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:43553 (local address 127.0.0.1:42462)
0811 14:40:05.271470 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.271518 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:43553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 14:40:05.272014 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.271864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:43553 (local address 127.0.0.1:42470)
0811 14:40:05.271931 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.271980 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:43553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 14:40:05.272154 6862 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.124:43553: Network error: Client connection negotiation failed: client connection to 127.2.103.124:43553: connect: Connection refused (error 111)
W20260811 14:40:05.296736 6998 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 14:40:05.296983 6998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:40:05.297001 6998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:40:05.297024 6998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:40:05.298244 6998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:40:05.298292 6998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:40:05.298307 6998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:40:05.298321 6998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:40:05.298333 6998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:40:05.299964 6998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:33991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:35185,127.2.103.125:37135,127.2.103.124:43553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.124.keytab
--principal=kudu/127.2.103.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.124:43553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.103.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:40:05.300267 6998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:40:05.300524 6998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:40:05.306517 7005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:40:05.306582 7006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:40:05.306528 7009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:40:05.307199 6998 server_base.cc:1034] running on GCE node
I20260811 14:40:05.307487 6998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:40:05.307886 6998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:40:05.309111 6998 hybrid_clock.cc:648] HybridClock initialized: now 1786459205309105 us; error 44 us; skew 500 ppm
Aug 11 14:40:05 dist-test-slave-twwt krb5kdc[6726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459205, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:40:05.312108 6998 init.cc:377] Logged in from keytab as kudu/127.2.103.124@KRBTEST.COM (short username kudu)
I20260811 14:40:05.312597 6998 webserver.cc:460] Webserver started at http://127.2.103.124:33123/ using document root <none> and password file <none>
I20260811 14:40:05.312904 6998 fs_manager.cc:359] Metadata directory not provided
I20260811 14:40:05.312981 6998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:40:05.313149 6998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:40:05.314268 6988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } attempt: 2
I20260811 14:40:05.314289 6998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-2/data/instance:
uuid: "ab53d1fc1a284222925d6f38e6d038af"
format_stamp: "Formatted at 2026-08-11 14:40:05 on dist-test-slave-twwt"
server_key: "3e326b0aa4c7131c2020edbef227ebbc"
server_key_iv: "670f83ea5c97e64d1b4de29f5eb0c750"
server_key_version: "encryptionkey@0"
I20260811 14:40:05.314738 6990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.314597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:43553 (local address 127.0.0.1:42484)
0811 14:40:05.314665 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.314699 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:43553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 14:40:05.314770 6998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-2/wal/instance:
uuid: "ab53d1fc1a284222925d6f38e6d038af"
format_stamp: "Formatted at 2026-08-11 14:40:05 on dist-test-slave-twwt"
server_key: "3e326b0aa4c7131c2020edbef227ebbc"
server_key_iv: "670f83ea5c97e64d1b4de29f5eb0c750"
server_key_version: "encryptionkey@0"
I20260811 14:40:05.315181 6990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.315050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:43553 (local address 127.0.0.1:42486)
0811 14:40:05.315123 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.315157 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:43553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 14:40:05.315327 6988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.124:43553: Network error: Client connection negotiation failed: client connection to 127.2.103.124:43553: connect: Connection refused (error 111)
I20260811 14:40:05.316460 6998 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:40:05.317370 7020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:40:05.317749 6998 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:40:05.317852 6998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-2/wal
uuid: "ab53d1fc1a284222925d6f38e6d038af"
format_stamp: "Formatted at 2026-08-11 14:40:05 on dist-test-slave-twwt"
server_key: "3e326b0aa4c7131c2020edbef227ebbc"
server_key_iv: "670f83ea5c97e64d1b4de29f5eb0c750"
server_key_version: "encryptionkey@0"
I20260811 14:40:05.317910 6998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:40:05.325438 6998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:40:05.327746 6998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:40:05.327867 6998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:40:05.332170 6998 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.124:43553
I20260811 14:40:05.332204 7097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.124:43553 every 8 connection(s)
I20260811 14:40:05.332600 6998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-2/data/info.pb
I20260811 14:40:05.333318 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 6998
I20260811 14:40:05.333400 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458919184199-2461-0/minicluster-data/master-2/wal/instance
I20260811 14:40:05.333618 2461 external_mini_cluster.cc:1428] Setting key 141841208eed39360a0ac794d80dc196
I20260811 14:40:05.333977 7099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:40:05.336724 7099 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 14:40:05 dist-test-slave-twwt krb5kdc[6726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.124@KRBTEST.COM
Aug 11 14:40:05 dist-test-slave-twwt krb5kdc[6726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459205, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.124@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:40:05.345598 7104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.334173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42496 (local address 127.2.103.124:43553)
0811 14:40:05.334383 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.334392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.334420 (+ 28us) server_negotiation.cc:407] Connection header received
0811 14:40:05.334964 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.334971 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.335023 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.335145 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.336085 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.337089 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.338133 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.338899 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.339010 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.341570 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:05.341576 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:05.341577 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:05.341584 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:05.343611 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.344167 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.344171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.344171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.344289 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.344548 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.344549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.344549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.344909 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:05.345039 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:05.345215 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 14:40:05.345357 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":158,"thread_start_us":112,"threads_started":1}
I20260811 14:40:05.346179 7105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.337482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:35185 (local address 127.0.0.1:44536)
0811 14:40:05.337723 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.337815 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0811 14:40:05.337985 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:40:05.338237 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:40:05.338240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:40:05.338645 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:40:05.338736 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:05.338739 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.339384 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.339385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:05.340111 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:05.340114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.340642 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.340643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:05.340699 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.341243 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:40:05.341254 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:40:05.342974 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:40:05.344891 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:05.344898 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:05.344900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:05.345117 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:05.345392 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:05.345392 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:05.345393 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:05.345460 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:05.345759 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:40:05.345764 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:40:05.345868 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 14:40:05.345961 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 14:40:05.346077 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":129,"threads_started":1}
I20260811 14:40:05.346190 6868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.337524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44536 (local address 127.2.103.126:35185)
0811 14:40:05.337604 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.337606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.337940 (+ 334us) server_negotiation.cc:407] Connection header received
0811 14:40:05.338085 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.338086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.338100 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.338158 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.338833 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.339309 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.340208 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.340522 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.340608 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.343071 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:05.343073 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:05.343073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:05.343081 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:05.344794 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.345237 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.345239 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.345239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.345319 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.345534 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.345535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.345535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.345683 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:05.345753 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:05.345996 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 14:40:05.346076 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 14:40:05.346807 7099 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:40:05.347333 6862 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } attempt: 2
Aug 11 14:40:05 dist-test-slave-twwt krb5kdc[6726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
Aug 11 14:40:05 dist-test-slave-twwt krb5kdc[6726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459205, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.124@KRBTEST.COM for kudu/127.2.103.125@KRBTEST.COM
Aug 11 14:40:05 dist-test-slave-twwt krb5kdc[6726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459204, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for kudu/127.2.103.124@KRBTEST.COM
I20260811 14:40:05.356020 7105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.347092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:37135 (local address 127.0.0.1:50460)
0811 14:40:05.347230 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.347238 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 14:40:05.347300 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:40:05.347422 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:40:05.347423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:40:05.347563 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:40:05.347597 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:05.347598 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.349339 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.349339 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:05.349943 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:05.349946 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.350489 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.350490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:05.350553 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.351016 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:40:05.351021 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:40:05.352657 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:40:05.354774 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:05.354775 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:05.354776 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:05.355006 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:05.355253 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:05.355254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:05.355254 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:05.355329 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:05.355635 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:40:05.355636 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:40:05.355685 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 14:40:05.355824 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 14:40:05.355917 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260811 14:40:05.356058 6995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.347176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50460 (local address 127.2.103.125:37135)
0811 14:40:05.347229 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.347231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.347288 (+ 57us) server_negotiation.cc:407] Connection header received
0811 14:40:05.347361 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.347363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.347372 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.347421 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.347700 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.348480 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.350045 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.350373 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.350412 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.352772 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:05.352774 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:05.352774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:05.352781 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:05.354661 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.355094 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.355096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.355096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.355162 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.355395 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.355396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.355396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.355537 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:05.355605 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:05.355841 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 14:40:05.355936 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:40:05.356472 7099 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:40:05.356966 6868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.346685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44552 (local address 127.2.103.126:35185)
0811 14:40:05.346744 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.346746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.346859 (+ 113us) server_negotiation.cc:407] Connection header received
0811 14:40:05.346938 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.346939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.346949 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.347001 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.347718 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.348480 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.349328 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.349611 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.349700 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.353510 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:05.353512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:05.353512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:05.353519 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:05.355425 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.355783 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.355784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.355784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.355945 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.356197 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.356198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.356198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.356348 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:05.356518 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:05.356776 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 14:40:05.356861 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 14:40:05.357077 7104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.347764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42504 (local address 127.2.103.124:43553)
0811 14:40:05.348600 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.348601 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.348608 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:40:05.348625 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.348628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.348642 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.348682 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.349041 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.349646 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.350591 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.351009 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.351087 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.353345 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:05.353346 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:05.353347 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:05.353354 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:05.355126 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.355643 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.355644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.355645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.355700 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.356054 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.356055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.356055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.356636 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:05.356710 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:05.356886 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 14:40:05.356977 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":817}
I20260811 14:40:05.357117 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.347561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:43553 (local address 127.0.0.1:42504)
0811 14:40:05.347644 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.347654 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:40:05.347691 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:40:05.348715 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:40:05.348716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:40:05.348921 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:40:05.348960 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:05.348961 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.349729 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.349730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:05.350505 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:05.350507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.351099 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.351100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:05.351167 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.351558 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:40:05.351563 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:40:05.353266 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:40:05.355231 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:05.355233 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:05.355233 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:05.355573 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:05.355783 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:05.355784 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:05.355784 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:05.355866 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:05.356703 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:40:05.356704 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:40:05.356752 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 14:40:05.356833 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 14:40:05.356920 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 14:40:05.358999 6862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c: Bootstrap starting.
I20260811 14:40:05.360213 6862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c: Neither blocks nor log segments found. Creating new log.
I20260811 14:40:05.360544 6862 log.cc:824] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c: Log is configured to *not* fsync() on all Append() calls
Aug 11 14:40:05 dist-test-slave-twwt krb5kdc[6726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459205, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.124@KRBTEST.COM for kudu/127.2.103.124@KRBTEST.COM
I20260811 14:40:05.361567 6862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c: No bootstrap required, opened a new log
Aug 11 14:40:05 dist-test-slave-twwt krb5kdc[6726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.125@KRBTEST.COM
I20260811 14:40:05.364028 6862 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef32f1c0f6c248209c898fa339fbc57c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } } peers { permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } } peers { permanent_uuid: "ab53d1fc1a284222925d6f38e6d038af" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } }
I20260811 14:40:05.364189 6862 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef32f1c0f6c248209c898fa339fbc57c, State: Initialized, Role: FOLLOWER
I20260811 14:40:05.364320 7105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.356777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:43553 (local address 127.0.0.1:42506)
0811 14:40:05.357001 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.357013 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:40:05.357056 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:40:05.357301 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:40:05.357302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:40:05.357449 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:40:05.357487 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:05.357489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.358294 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.358295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:05.358948 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:05.358951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.359352 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.359353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:05.359403 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.359835 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:40:05.359839 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:40:05.361220 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:40:05.363097 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:05.363098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:05.363098 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:05.363320 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:05.363546 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:05.363547 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:05.363547 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:05.363614 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:05.363988 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:40:05.363990 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:40:05.364036 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 14:40:05.364120 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 14:40:05.364225 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204}
I20260811 14:40:05.364277 6862 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef32f1c0f6c248209c898fa339fbc57c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } } peers { permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } } peers { permanent_uuid: "ab53d1fc1a284222925d6f38e6d038af" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } }
I20260811 14:40:05.364575 7111 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef32f1c0f6c248209c898fa339fbc57c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } } peers { permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } } peers { permanent_uuid: "ab53d1fc1a284222925d6f38e6d038af" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } } }
I20260811 14:40:05.364643 7104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.356849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42506 (local address 127.2.103.124:43553)
0811 14:40:05.357163 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.357165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.357174 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:40:05.357197 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.357197 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.357209 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.357544 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.357562 (+ 18us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.358208 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.359030 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.359262 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.359353 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.361323 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:05.361325 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:05.361325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:05.361333 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:05.363004 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.363405 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.363407 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.363407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.363467 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.363685 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.363686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.363686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.363901 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:05.363978 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:05.364148 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 14:40:05.364549 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":297,"thread_start_us":110,"threads_started":1}
I20260811 14:40:05.364650 6862 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:40:05.364656 7111 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:40:05.366420 7099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af: Bootstrap starting.
I20260811 14:40:05.367431 7099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af: Neither blocks nor log segments found. Creating new log.
I20260811 14:40:05.367726 7099 log.cc:824] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af: Log is configured to *not* fsync() on all Append() calls
I20260811 14:40:05.368567 7099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af: No bootstrap required, opened a new log
I20260811 14:40:05.369416 6995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.358409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50470 (local address 127.2.103.125:37135)
0811 14:40:05.358446 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.358448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.358458 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:40:05.358479 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.358480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.358488 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.358545 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.359178 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.359929 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.360880 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.361218 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.361288 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.364068 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:05.364070 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:05.364070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:05.364077 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:05.366062 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.368393 (+ 2331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.368395 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.368395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.368458 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.368725 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.368726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.368727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.368874 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:05.369018 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:05.369228 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 14:40:05.369324 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 14:40:05.370589 7099 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef32f1c0f6c248209c898fa339fbc57c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } } peers { permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } } peers { permanent_uuid: "ab53d1fc1a284222925d6f38e6d038af" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } }
I20260811 14:40:05.370829 7099 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab53d1fc1a284222925d6f38e6d038af, State: Initialized, Role: FOLLOWER
I20260811 14:40:05.370932 7099 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef32f1c0f6c248209c898fa339fbc57c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } } peers { permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } } peers { permanent_uuid: "ab53d1fc1a284222925d6f38e6d038af" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } }
I20260811 14:40:05.371219 7135 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef32f1c0f6c248209c898fa339fbc57c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } } peers { permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } } peers { permanent_uuid: "ab53d1fc1a284222925d6f38e6d038af" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } } }
I20260811 14:40:05.371287 7099 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:40:05.371357 7135 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:40:05.369300 6745 master_runner.cc:186] Error getting master registration for 127.2.103.125:37135: OK, instance_id { permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" instance_seqno: 1786459205165464 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 14:40:05.371815 6745 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 14:40:05.366715 7131 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:40:05.372077 7131 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:40:05.373564 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260811 14:40:05.373715 7152 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:40:05.373819 7152 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:40:05.374598 6810 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ef32f1c0f6c248209c898fa339fbc57c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44552
I20260811 14:40:05.374768 6810 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 14:40:05.374840 6810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:40:05.376214 6810 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef32f1c0f6c248209c898fa339fbc57c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } } peers { permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } } peers { permanent_uuid: "ab53d1fc1a284222925d6f38e6d038af" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } }
I20260811 14:40:05.376457 6810 leader_election.cc:288] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [CANDIDATE]: Term 1 election: Requested vote from peers ceab8363121e41d7ab17360eb8397ec3 (127.2.103.125:37135), ab53d1fc1a284222925d6f38e6d038af (127.2.103.124:43553)
I20260811 14:40:05.376950 7065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ef32f1c0f6c248209c898fa339fbc57c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ab53d1fc1a284222925d6f38e6d038af"
I20260811 14:40:05.377070 7065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:40:05.378033 7065 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef32f1c0f6c248209c898fa339fbc57c in term 1.
I20260811 14:40:05.378249 6774 leader_election.cc:302] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab53d1fc1a284222925d6f38e6d038af, ef32f1c0f6c248209c898fa339fbc57c; no voters:
I20260811 14:40:05.375381 6998 master_runner.cc:186] Error getting master registration for 127.2.103.125:37135: OK, instance_id { permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" instance_seqno: 1786459205165464 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 14:40:05.378353 6998 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 14:40:05.378407 7111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:40:05.378595 7111 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [term 1 LEADER]: Becoming Leader. State: Replica: ef32f1c0f6c248209c898fa339fbc57c, State: Running, Role: LEADER
I20260811 14:40:05.378727 7111 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef32f1c0f6c248209c898fa339fbc57c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } } peers { permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } } peers { permanent_uuid: "ab53d1fc1a284222925d6f38e6d038af" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } }
I20260811 14:40:05.379207 7111 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef32f1c0f6c248209c898fa339fbc57c. Latest consensus state: current_term: 1 leader_uuid: "ef32f1c0f6c248209c898fa339fbc57c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef32f1c0f6c248209c898fa339fbc57c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } } peers { permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } } peers { permanent_uuid: "ab53d1fc1a284222925d6f38e6d038af" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } } }
I20260811 14:40:05.379307 7111 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [sys.catalog]: This master's current role is: LEADER
I20260811 14:40:05.387706 7157 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:40:05.388000 7157 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:40:05.391836 7065 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af [term 1 FOLLOWER]: Refusing update from remote peer ef32f1c0f6c248209c898fa339fbc57c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:40:05.392411 7154 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab53d1fc1a284222925d6f38e6d038af" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:40:05.394083 7135 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef32f1c0f6c248209c898fa339fbc57c. Latest consensus state: current_term: 1 leader_uuid: "ef32f1c0f6c248209c898fa339fbc57c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef32f1c0f6c248209c898fa339fbc57c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } } peers { permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } } peers { permanent_uuid: "ab53d1fc1a284222925d6f38e6d038af" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } } }
I20260811 14:40:05.394197 7135 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:40:05.403352 6868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.387453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44558 (local address 127.2.103.126:35185)
0811 14:40:05.387519 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.387521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.389313 (+ 1792us) server_negotiation.cc:407] Connection header received
0811 14:40:05.389402 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.389403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.389416 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.389494 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.390627 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.391371 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.395719 (+ 4348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.396042 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.396087 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.398637 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:05.398639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:05.398639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:05.398647 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:05.401113 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.401532 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.401534 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.401534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.401624 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.402678 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.402679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.402679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.402847 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:05.402969 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:05.403151 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 14:40:05.403241 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 14:40:05.404059 7104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.387187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42512 (local address 127.2.103.124:43553)
0811 14:40:05.387245 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.387248 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.389469 (+ 2221us) server_negotiation.cc:407] Connection header received
0811 14:40:05.389542 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.389544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.389557 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.389616 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.390521 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.391317 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.392410 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.392738 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.392842 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.395691 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:05.395693 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:05.395693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:05.395702 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:05.397628 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.402712 (+ 5084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.402714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.402714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.402783 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.403405 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.403407 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.403407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.403559 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:05.403643 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:05.403855 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 14:40:05.403962 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 14:40:05.406402 6988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } attempt: 3
I20260811 14:40:05.409258 7154 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef32f1c0f6c248209c898fa339fbc57c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef32f1c0f6c248209c898fa339fbc57c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } } peers { permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } } peers { permanent_uuid: "ab53d1fc1a284222925d6f38e6d038af" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } } }
I20260811 14:40:05.409377 7154 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [sys.catalog]: This master's current role is: LEADER
I20260811 14:40:05.410763 7171 mvcc.cc:205] Tried to move back new op lower bound from 7317336905277276160 to 7317336905232109568. Current Snapshot: MvccSnapshot[applied={T|T < 7317336905277276160}]
I20260811 14:40:05.410945 7135 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef32f1c0f6c248209c898fa339fbc57c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef32f1c0f6c248209c898fa339fbc57c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } } peers { permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } } peers { permanent_uuid: "ab53d1fc1a284222925d6f38e6d038af" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } } }
I20260811 14:40:05.411031 7135 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af [sys.catalog]: This master's current role is: FOLLOWER
Aug 11 14:40:05 dist-test-slave-twwt krb5kdc[6726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459205, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.125@KRBTEST.COM for kudu/127.2.103.124@KRBTEST.COM
I20260811 14:40:05.412526 7167 mvcc.cc:205] Tried to move back new op lower bound from 7317336905277276160 to 7317336905232109568. Current Snapshot: MvccSnapshot[applied={T|T < 7317336905277276160}]
I20260811 14:40:05.413904 6995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.387066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50482 (local address 127.2.103.125:37135)
0811 14:40:05.387418 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.387420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.389624 (+ 2204us) server_negotiation.cc:407] Connection header received
0811 14:40:05.389728 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.389730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.389744 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.389776 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.399570 (+ 9794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.400342 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.404679 (+ 4337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.405045 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.405094 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.406801 (+ 1707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:05.406803 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:05.406803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:05.406812 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:05.410265 (+ 3453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.412988 (+ 2723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.412990 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.412990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.413055 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.413276 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.413277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.413277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.413408 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:05.413490 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:05.413705 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 14:40:05.413792 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325}
I20260811 14:40:05.415215 7157 catalog_manager.cc:1347] Generated new cluster ID: b448365bd4794ea889b4cd0fce010d91
I20260811 14:40:05.415272 7157 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:40:05.416065 6990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.406711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:43553 (local address 127.0.0.1:42514)
0811 14:40:05.406806 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:05.406818 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:40:05.406859 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:40:05.407191 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:40:05.407192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:40:05.407695 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:40:05.407745 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:05.407748 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.408681 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.408682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:05.409527 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:05.409530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.410284 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.410285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:05.410341 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.410866 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:40:05.410872 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:40:05.412384 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:40:05.414968 (+ 2584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:05.414969 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:05.414970 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:05.415193 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:05.415412 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:05.415413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:05.415414 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:05.415470 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:05.415739 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:40:05.415740 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:40:05.415782 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 14:40:05.415842 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 14:40:05.415951 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 14:40:05.416407 7104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:05.406940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42514 (local address 127.2.103.124:43553)
0811 14:40:05.407046 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.407048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.407056 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:40:05.407076 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.407077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.407089 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.407141 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.407855 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.408601 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.409778 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:05.410102 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:05.410166 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:05.412473 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:05.412474 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:05.412475 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:05.412498 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:05.414878 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.415284 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.415285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.415285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.415343 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:05.415535 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:05.415535 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:05.415536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:05.415659 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:05.415715 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:05.416242 (+ 527us) server_negotiation.cc:299] Negotiation successful
0811 14:40:05.416320 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 14:40:05.418031 6988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3: Bootstrap starting.
I20260811 14:40:05.418824 6988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3: Neither blocks nor log segments found. Creating new log.
I20260811 14:40:05.419139 6988 log.cc:824] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3: Log is configured to *not* fsync() on all Append() calls
I20260811 14:40:05.427486 6988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3: No bootstrap required, opened a new log
I20260811 14:40:05.429670 6988 raft_consensus.cc:348] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef32f1c0f6c248209c898fa339fbc57c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } } peers { permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } } peers { permanent_uuid: "ab53d1fc1a284222925d6f38e6d038af" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } }
I20260811 14:40:05.429852 6988 raft_consensus.cc:729] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceab8363121e41d7ab17360eb8397ec3, State: Initialized, Role: FOLLOWER
I20260811 14:40:05.429951 6988 consensus_queue.cc:260] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef32f1c0f6c248209c898fa339fbc57c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } } peers { permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } } peers { permanent_uuid: "ab53d1fc1a284222925d6f38e6d038af" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } }
I20260811 14:40:05.430334 6988 sys_catalog.cc:564] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:40:05.431109 6944 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:40:05.431910 7157 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:40:05.432112 6944 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3 [term 1 FOLLOWER]: Refusing update from remote peer ef32f1c0f6c248209c898fa339fbc57c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:40:05.432415 7154 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:40:05.432857 7157 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:40:05.434826 6946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ef32f1c0f6c248209c898fa339fbc57c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ceab8363121e41d7ab17360eb8397ec3"
I20260811 14:40:05.436677 7187 sys_catalog.cc:455] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef32f1c0f6c248209c898fa339fbc57c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef32f1c0f6c248209c898fa339fbc57c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } } peers { permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } } peers { permanent_uuid: "ab53d1fc1a284222925d6f38e6d038af" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } } }
I20260811 14:40:05.436780 7187 sys_catalog.cc:458] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:40:05.436836 7187 sys_catalog.cc:455] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef32f1c0f6c248209c898fa339fbc57c. Latest consensus state: current_term: 1 leader_uuid: "ef32f1c0f6c248209c898fa339fbc57c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef32f1c0f6c248209c898fa339fbc57c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } } peers { permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } } peers { permanent_uuid: "ab53d1fc1a284222925d6f38e6d038af" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } } }
I20260811 14:40:05.436884 7187 sys_catalog.cc:458] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:40:05.436928 7187 sys_catalog.cc:455] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef32f1c0f6c248209c898fa339fbc57c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef32f1c0f6c248209c898fa339fbc57c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } } peers { permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } } peers { permanent_uuid: "ab53d1fc1a284222925d6f38e6d038af" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } } }
I20260811 14:40:05.436973 7187 sys_catalog.cc:458] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:40:05.437739 7185 mvcc.cc:205] Tried to move back new op lower bound from 7317336905442390016 to 7317336905232109568. Current Snapshot: MvccSnapshot[applied={T|T < 7317336905277276160}]
I20260811 14:40:05.437942 6946 raft_consensus.cc:2396] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ef32f1c0f6c248209c898fa339fbc57c for term 1 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
W20260811 14:40:05.439878 7199 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:40:05.439950 7199 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:40:05.440797 7154 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef32f1c0f6c248209c898fa339fbc57c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef32f1c0f6c248209c898fa339fbc57c" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 35185 } } peers { permanent_uuid: "ceab8363121e41d7ab17360eb8397ec3" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 37135 } } peers { permanent_uuid: "ab53d1fc1a284222925d6f38e6d038af" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 43553 } } }
I20260811 14:40:05.440919 7154 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c [sys.catalog]: This master's current role is: LEADER
I20260811 14:40:05.441605 7157 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef32f1c0f6c248209c898fa339fbc57c: Generated new TSK 0
I20260811 14:40:05.441835 7157 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 14:40:05.470659 6995 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:05.469221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50490 (local address 127.2.103.125:37135)
0811 14:40:05.469271 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.469273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.469919 (+ 646us) server_negotiation.cc:407] Connection header received
0811 14:40:05.470027 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.470028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.470041 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.470070 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.470565 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50490: BlockingRecv error: recv got EOF from 127.0.0.1:50490 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260811 14:40:05.471530 7104 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:05.469628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42516 (local address 127.2.103.124:43553)
0811 14:40:05.469669 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.469671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.470952 (+ 1281us) server_negotiation.cc:407] Connection header received
0811 14:40:05.471099 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.471100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.471110 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.471137 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.471469 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42516: BlockingRecv error: recv got EOF from 127.0.0.1:42516 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260811 14:40:05.473066 6868 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:05.471975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44568 (local address 127.2.103.126:35185)
0811 14:40:05.472019 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.472021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.472384 (+ 363us) server_negotiation.cc:407] Connection header received
0811 14:40:05.472530 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.472531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.472541 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.472571 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.473000 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44568: BlockingRecv error: recv got EOF from 127.0.0.1:44568 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260811 14:40:05.477784 6995 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:05.474939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50494 (local address 127.2.103.125:37135)
0811 14:40:05.474978 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.474980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.477167 (+ 2187us) server_negotiation.cc:407] Connection header received
0811 14:40:05.477295 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.477297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.477319 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.477354 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.477730 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50494: BlockingRecv error: recv got EOF from 127.0.0.1:50494 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260811 14:40:05.481097 6995 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:05.480196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50498 (local address 127.2.103.125:37135)
0811 14:40:05.480234 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.480236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.480517 (+ 281us) server_negotiation.cc:407] Connection header received
0811 14:40:05.480621 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.480622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.480634 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.480665 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.481043 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50498: BlockingRecv error: recv got EOF from 127.0.0.1:50498 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260811 14:40:05.485103 6995 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:05.484228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50502 (local address 127.2.103.125:37135)
0811 14:40:05.484270 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.484272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.484531 (+ 259us) server_negotiation.cc:407] Connection header received
0811 14:40:05.484632 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.484633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.484645 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.484674 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.485049 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50502: BlockingRecv error: recv got EOF from 127.0.0.1:50502 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260811 14:40:05.491590 6995 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:05.490459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50512 (local address 127.2.103.125:37135)
0811 14:40:05.490508 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.490510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.490938 (+ 428us) server_negotiation.cc:407] Connection header received
0811 14:40:05.491061 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.491062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.491075 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.491107 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.491527 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50512: BlockingRecv error: recv got EOF from 127.0.0.1:50512 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260811 14:40:05.501966 6995 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:05.500934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50522 (local address 127.2.103.125:37135)
0811 14:40:05.500982 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.500985 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.501297 (+ 312us) server_negotiation.cc:407] Connection header received
0811 14:40:05.501413 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.501414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.501428 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.501460 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.501888 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50522: BlockingRecv error: recv got EOF from 127.0.0.1:50522 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260811 14:40:05.520505 6995 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:05.519486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50526 (local address 127.2.103.125:37135)
0811 14:40:05.519538 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.519540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.519846 (+ 306us) server_negotiation.cc:407] Connection header received
0811 14:40:05.519960 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.519962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.519975 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.520008 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.520441 (+ 433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50526: BlockingRecv error: recv got EOF from 127.0.0.1:50526 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260811 14:40:05.555051 6995 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:05.553927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50536 (local address 127.2.103.125:37135)
0811 14:40:05.553975 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.553979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.554303 (+ 324us) server_negotiation.cc:407] Connection header received
0811 14:40:05.554451 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.554452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.554466 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.554512 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.554984 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50536: BlockingRecv error: recv got EOF from 127.0.0.1:50536 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260811 14:40:05.621598 6995 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:05.620693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50552 (local address 127.2.103.125:37135)
0811 14:40:05.620755 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.620757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.620890 (+ 133us) server_negotiation.cc:407] Connection header received
0811 14:40:05.621028 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.621029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.621044 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.621140 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.621520 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50552: BlockingRecv error: recv got EOF from 127.0.0.1:50552 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260811 14:40:05.751752 6995 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:05.750946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50558 (local address 127.2.103.125:37135)
0811 14:40:05.751046 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:05.751048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:05.751143 (+ 95us) server_negotiation.cc:407] Connection header received
0811 14:40:05.751234 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:05.751235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:05.751248 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:05.751303 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:05.751679 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50558: BlockingRecv error: recv got EOF from 127.0.0.1:50558 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 14:40:06.009940 6995 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:06.009130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50568 (local address 127.2.103.125:37135)
0811 14:40:06.009221 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:06.009223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:06.009299 (+ 76us) server_negotiation.cc:407] Connection header received
0811 14:40:06.009406 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:06.009408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:06.009419 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:06.009500 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:06.009870 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50568: BlockingRecv error: recv got EOF from 127.0.0.1:50568 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 14:40:06.374341 7152 catalog_manager.cc:1259] Loaded cluster ID: b448365bd4794ea889b4cd0fce010d91
I20260811 14:40:06.374500 7152 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af: loading cluster ID for follower catalog manager: success
I20260811 14:40:06.375388 7152 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af: acquiring CA information for follower catalog manager: success
I20260811 14:40:06.375715 7152 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ab53d1fc1a284222925d6f38e6d038af: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 14:40:06.440459 7199 catalog_manager.cc:1259] Loaded cluster ID: b448365bd4794ea889b4cd0fce010d91
I20260811 14:40:06.440577 7199 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3: loading cluster ID for follower catalog manager: success
I20260811 14:40:06.441429 7199 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3: acquiring CA information for follower catalog manager: success
I20260811 14:40:06.441761 7199 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ceab8363121e41d7ab17360eb8397ec3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 14:40:06.526207 7309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:06.523296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50584 (local address 127.2.103.125:37135)
0811 14:40:06.523433 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:06.523435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:06.523472 (+ 37us) server_negotiation.cc:407] Connection header received
0811 14:40:06.523547 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:06.523548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:06.523651 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:06.523762 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:40:06.524018 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:06.524585 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:06.525374 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:06.525573 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:06.525632 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:06.525847 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:40:06.525962 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:40:06.526023 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:06.526064 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 14:40:06.526083 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":82,"threads_started":1}
I20260811 14:40:06.530858 7316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:06.528409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42518 (local address 127.2.103.124:43553)
0811 14:40:06.528537 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:06.528540 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:06.528554 (+ 14us) server_negotiation.cc:407] Connection header received
0811 14:40:06.528591 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:06.528592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:06.528628 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:06.528699 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:40:06.528839 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:06.529309 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:06.529998 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:06.530250 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:06.530303 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:06.530455 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:40:06.530556 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:40:06.530675 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:06.530711 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 14:40:06.530745 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260811 14:40:06.535786 7316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:06.532778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42532 (local address 127.2.103.124:43553)
0811 14:40:06.532817 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:06.532819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:06.533116 (+ 297us) server_negotiation.cc:407] Connection header received
0811 14:40:06.533204 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:06.533205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:06.533215 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:06.533272 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:40:06.533377 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:06.533895 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:06.534957 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:06.535257 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:06.535328 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:06.535444 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:40:06.535473 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:40:06.535524 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:06.535638 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 14:40:06.535663 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:40:06.536156 7309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:06.532842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50594 (local address 127.2.103.125:37135)
0811 14:40:06.532879 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:06.532881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:06.533109 (+ 228us) server_negotiation.cc:407] Connection header received
0811 14:40:06.533223 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:06.533223 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:06.533233 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:06.533263 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:40:06.533434 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:06.534171 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:06.535274 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:06.535591 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:06.535653 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:06.535872 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:40:06.535909 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:40:06.535940 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:06.536048 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 14:40:06.536071 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 14:40:06.536368 7333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:06.532701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44582 (local address 127.2.103.126:35185)
0811 14:40:06.532830 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:06.532832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:06.533038 (+ 206us) server_negotiation.cc:407] Connection header received
0811 14:40:06.533152 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:06.533153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:06.533222 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:06.533300 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:40:06.533483 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:06.534224 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:06.535159 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:06.535526 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:06.535597 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:06.535889 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:40:06.536003 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:40:06.536095 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:06.536153 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 14:40:06.536176 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":70,"threads_started":1}
I20260811 14:40:06.538244 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 6745
I20260811 14:40:06.555392 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 6874
I20260811 14:40:06.563127 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 6998
2026-08-11T14:40:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3142 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:40:06 dist-test-slave-twwt krb5kdc[7338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:40:06 dist-test-slave-twwt krb5kdc[7338](info): set up 2 sockets
Aug 11 14:40:06 dist-test-slave-twwt krb5kdc[7338](info): commencing operation
krb5kdc: starting...
W20260811 14:40:07.839299 2461 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/test-user.keytab.
Aug 11 14:40:07 dist-test-slave-twwt krb5kdc[7338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:40:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:40:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.103.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
Entry for principal kudu/127.2.103.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab.
I20260811 14:40:07.974160 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:46519
--webserver_interface=127.2.103.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:46613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:46519,127.2.103.125:42965,127.2.103.124:44157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:40:08.060268 7355 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 14:40:08.060504 7355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:40:08.060523 7355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:40:08.060544 7355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:40:08.061641 7355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:40:08.061677 7355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:40:08.061692 7355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:40:08.061705 7355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:40:08.061753 7355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:40:08.063438 7355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:46613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:46519,127.2.103.125:42965,127.2.103.124:44157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.126.keytab
--principal=kudu/127.2.103.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.126:46519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.103.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:40:08.063684 7355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:40:08.063885 7355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:40:08.069077 7368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:40:08.069160 7355 server_base.cc:1034] running on GCE node
W20260811 14:40:08.069118 7362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:40:08.069120 7361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:40:08.069589 7355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:40:08.069977 7355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:40:08.071146 7355 hybrid_clock.cc:648] HybridClock initialized: now 1786459208071128 us; error 43 us; skew 500 ppm
Aug 11 14:40:08 dist-test-slave-twwt krb5kdc[7338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459208, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:40:08.073808 7355 init.cc:377] Logged in from keytab as kudu/127.2.103.126@KRBTEST.COM (short username kudu)
I20260811 14:40:08.074281 7355 webserver.cc:460] Webserver started at http://127.2.103.126:33071/ using document root <none> and password file <none>
I20260811 14:40:08.074528 7355 fs_manager.cc:359] Metadata directory not provided
I20260811 14:40:08.074599 7355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:40:08.074786 7355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:40:08.075902 7355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-0/data/instance:
uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5"
format_stamp: "Formatted at 2026-08-11 14:40:08 on dist-test-slave-twwt"
server_key: "839b86b57b0e869ed9137ff0cdf54197"
server_key_iv: "ac7ab24b79c2f0991ef565497cc1ab08"
server_key_version: "encryptionkey@0"
I20260811 14:40:08.076328 7355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-0/wal/instance:
uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5"
format_stamp: "Formatted at 2026-08-11 14:40:08 on dist-test-slave-twwt"
server_key: "839b86b57b0e869ed9137ff0cdf54197"
server_key_iv: "ac7ab24b79c2f0991ef565497cc1ab08"
server_key_version: "encryptionkey@0"
I20260811 14:40:08.077728 7355 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:40:08.078521 7379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:40:08.078740 7355 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:40:08.078892 7355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-0/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-0/wal
uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5"
format_stamp: "Formatted at 2026-08-11 14:40:08 on dist-test-slave-twwt"
server_key: "839b86b57b0e869ed9137ff0cdf54197"
server_key_iv: "ac7ab24b79c2f0991ef565497cc1ab08"
server_key_version: "encryptionkey@0"
I20260811 14:40:08.078979 7355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:40:08.087023 7355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:40:08.089525 7355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:40:08.089697 7355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:40:08.094135 7355 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.126:46519
I20260811 14:40:08.094175 7458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.126:46519 every 8 connection(s)
I20260811 14:40:08.094558 7355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-0/data/info.pb
I20260811 14:40:08.095892 7461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:40:08.097568 7461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:40:08.100359 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 7355
I20260811 14:40:08.100463 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-0/wal/instance
I20260811 14:40:08.100646 2461 external_mini_cluster.cc:1428] Setting key a9b1ac9f5124acb4f33955dae7df6bbd
Aug 11 14:40:08 dist-test-slave-twwt krb5kdc[7338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459208, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:40:08.108777 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.098258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46519 (local address 127.0.0.1:58876)
0811 14:40:08.098419 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.098468 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 14:40:08.099087 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:40:08.099379 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:40:08.099382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:40:08.099640 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:40:08.099806 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:08.099810 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.101308 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.101309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:08.102022 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:08.102024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.102809 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.102811 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:08.102909 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.103477 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:40:08.103489 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:40:08.105187 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:40:08.107173 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:08.107180 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:08.107183 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:08.107395 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:08.107672 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:08.107673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:08.107673 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:08.107761 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:08.108138 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:40:08.108144 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:40:08.108210 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 14:40:08.108324 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 14:40:08.108584 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":106,"thread_start_us":71,"threads_started":1}
I20260811 14:40:08.108780 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.098354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58876 (local address 127.2.103.126:46519)
0811 14:40:08.098489 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.098492 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.098585 (+ 93us) server_negotiation.cc:407] Connection header received
0811 14:40:08.099240 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.099242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.099259 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.099340 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.099929 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.101195 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.102111 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.102700 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.102789 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.105300 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:08.105308 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:08.105308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:08.105322 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:08.107070 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.107502 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.107505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.107505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.107591 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.107834 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.107835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.107835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.108065 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:08.108161 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:08.108360 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 14:40:08.108514 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260811 14:40:08.109575 7461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.103.125@KRBTEST.COM; defaulting to no policy
I20260811 14:40:08.109992 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.109862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:42965 (local address 127.0.0.1:42388)
0811 14:40:08.109914 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.109968 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.125:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 14:40:08.110158 7381 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.103.125:42965: connect: Connection refused (error 111)
I20260811 14:40:08.110700 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.110542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:42965 (local address 127.0.0.1:42392)
0811 14:40:08.110593 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.110677 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.125:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 14:40:08.110843 7461 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.125:42965: Network error: Client connection negotiation failed: client connection to 127.2.103.125:42965: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.125.keytab.
Entry for principal kudu/127.2.103.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.125.keytab.
I20260811 14:40:08.130200 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.125:42965
--webserver_interface=127.2.103.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.125.keytab
--principal=kudu/127.2.103.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:46613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:46519,127.2.103.125:42965,127.2.103.124:44157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:40:08.160130 7461 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } attempt: 1
I20260811 14:40:08.160738 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.160577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:42965 (local address 127.0.0.1:42402)
0811 14:40:08.160652 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.160701 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.125:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 14:40:08.161164 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.161056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:42965 (local address 127.0.0.1:42416)
0811 14:40:08.161110 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.161143 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.125:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 14:40:08.161270 7461 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.125:42965: Network error: Client connection negotiation failed: client connection to 127.2.103.125:42965: connect: Connection refused (error 111)
W20260811 14:40:08.218940 7476 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 14:40:08.219211 7476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:40:08.219244 7476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:40:08.219288 7476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:40:08.220459 7476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:40:08.220533 7476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:40:08.220557 7476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:40:08.220578 7476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:40:08.220609 7476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:40:08.222173 7476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:46613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:46519,127.2.103.125:42965,127.2.103.124:44157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.125.keytab
--principal=kudu/127.2.103.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.125:42965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.103.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:40:08.222472 7476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:40:08.222736 7476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:40:08.227739 7486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:40:08.227805 7485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:40:08.227805 7490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:40:08.227898 7476 server_base.cc:1034] running on GCE node
I20260811 14:40:08.228298 7476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:40:08.228610 7476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:40:08.229417 7461 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } attempt: 2
I20260811 14:40:08.229782 7476 hybrid_clock.cc:648] HybridClock initialized: now 1786459208229755 us; error 35 us; skew 500 ppm
I20260811 14:40:08.229852 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.229710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:42965 (local address 127.0.0.1:42424)
0811 14:40:08.229776 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.229817 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.125:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 14:40:08.230306 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.230169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:42965 (local address 127.0.0.1:42438)
0811 14:40:08.230249 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.230280 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.125:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 14:40:08.230463 7461 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.125:42965: Network error: Client connection negotiation failed: client connection to 127.2.103.125:42965: connect: Connection refused (error 111)
Aug 11 14:40:08 dist-test-slave-twwt krb5kdc[7338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459208, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:40:08.232637 7476 init.cc:377] Logged in from keytab as kudu/127.2.103.125@KRBTEST.COM (short username kudu)
I20260811 14:40:08.233141 7476 webserver.cc:460] Webserver started at http://127.2.103.125:40485/ using document root <none> and password file <none>
I20260811 14:40:08.233385 7476 fs_manager.cc:359] Metadata directory not provided
I20260811 14:40:08.233464 7476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:40:08.233628 7476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:40:08.234808 7476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-1/data/instance:
uuid: "4e818811921b444c800c9df37916259d"
format_stamp: "Formatted at 2026-08-11 14:40:08 on dist-test-slave-twwt"
server_key: "af32adb0bf86049c6c6ffac61ba5bf22"
server_key_iv: "4515d7c4953079e88d325312435b14e1"
server_key_version: "encryptionkey@0"
I20260811 14:40:08.235196 7476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-1/wal/instance:
uuid: "4e818811921b444c800c9df37916259d"
format_stamp: "Formatted at 2026-08-11 14:40:08 on dist-test-slave-twwt"
server_key: "af32adb0bf86049c6c6ffac61ba5bf22"
server_key_iv: "4515d7c4953079e88d325312435b14e1"
server_key_version: "encryptionkey@0"
I20260811 14:40:08.236604 7476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:40:08.237442 7499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:40:08.237639 7476 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:40:08.237753 7476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-1/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-1/wal
uuid: "4e818811921b444c800c9df37916259d"
format_stamp: "Formatted at 2026-08-11 14:40:08 on dist-test-slave-twwt"
server_key: "af32adb0bf86049c6c6ffac61ba5bf22"
server_key_iv: "4515d7c4953079e88d325312435b14e1"
server_key_version: "encryptionkey@0"
I20260811 14:40:08.237833 7476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:40:08.252765 7476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:40:08.254956 7476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:40:08.255143 7476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:40:08.259384 7476 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.125:42965
I20260811 14:40:08.259423 7575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.125:42965 every 8 connection(s)
I20260811 14:40:08.259816 7476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-1/data/info.pb
I20260811 14:40:08.260892 7577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:40:08.262713 7577 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:40:08.266711 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 7476
I20260811 14:40:08.266815 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-1/wal/instance
I20260811 14:40:08.267059 2461 external_mini_cluster.cc:1428] Setting key 8518879a95ac2eb64645d0ec318f9508
Aug 11 14:40:08 dist-test-slave-twwt krb5kdc[7338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459208, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.125@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:40:08.273343 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.263648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58890 (local address 127.2.103.126:46519)
0811 14:40:08.263722 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.263724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.263961 (+ 237us) server_negotiation.cc:407] Connection header received
0811 14:40:08.264491 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.264492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.264514 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.264599 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.265249 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.265755 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.266997 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.267264 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.267330 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.270251 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:08.270253 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:08.270253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:08.270261 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:08.271714 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.272270 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.272271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.272271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.272325 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.272623 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.272624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.272624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.272740 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:08.272791 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:08.273201 (+ 410us) server_negotiation.cc:299] Negotiation successful
0811 14:40:08.273275 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 14:40:08.273674 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.263630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46519 (local address 127.0.0.1:58890)
0811 14:40:08.263820 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.263873 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 14:40:08.264381 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:40:08.264683 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:40:08.264686 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:40:08.264949 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:40:08.265149 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:08.265154 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.265859 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.265861 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:08.266907 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:08.266911 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.267377 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.267378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:08.267536 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.268087 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:40:08.268099 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:40:08.270149 (+ 2050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:40:08.271809 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:08.271818 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:08.271821 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:08.272190 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:08.272400 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:08.272401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:08.272402 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:08.272554 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:08.272810 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:40:08.272817 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:40:08.273033 (+ 216us) client_negotiation.cc:769] Sending connection context
0811 14:40:08.273174 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 14:40:08.273376 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":87,"threads_started":1}
I20260811 14:40:08.274394 7577 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.103.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.103.124@KRBTEST.COM" created.
Aug 11 14:40:08 dist-test-slave-twwt krb5kdc[7338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459208, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.125@KRBTEST.COM for kudu/127.2.103.125@KRBTEST.COM
I20260811 14:40:08.284448 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.274712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:42965 (local address 127.0.0.1:42440)
0811 14:40:08.274769 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.274781 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:40:08.274826 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:40:08.275195 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:40:08.275196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:40:08.275334 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:40:08.275392 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:08.275394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.276734 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.276735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:08.277324 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:08.277326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.277797 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.277798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:08.277858 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.278277 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:40:08.278282 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:40:08.280914 (+ 2632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:40:08.283248 (+ 2334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:08.283249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:08.283250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:08.283420 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:08.283674 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:08.283675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:08.283675 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:08.283736 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:08.284092 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:40:08.284093 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:40:08.284133 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 14:40:08.284224 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 14:40:08.284337 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 14:40:08.284564 7582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.274803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42440 (local address 127.2.103.125:42965)
0811 14:40:08.274991 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.274996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.275023 (+ 27us) server_negotiation.cc:407] Connection header received
0811 14:40:08.275050 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.275053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.275069 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.275163 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.275485 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.276629 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.277419 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.277728 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.277788 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.281005 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:08.281019 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:08.281019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:08.281027 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:08.283129 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.283506 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.283511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.283511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.283580 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.283840 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.283841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.283848 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.284015 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:08.284115 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:08.284259 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 14:40:08.284434 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":154,"thread_start_us":96,"threads_started":1}
I20260811 14:40:08.285456 7577 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:40:08.285831 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.285697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:44157 (local address 127.0.0.1:54570)
0811 14:40:08.285750 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.285798 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:44157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 14:40:08.285976 7500 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.103.124:44157: connect: Connection refused (error 111)
I20260811 14:40:08.286397 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.286289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:44157 (local address 127.0.0.1:54574)
0811 14:40:08.286333 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.286371 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:44157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 14:40:08.286514 7577 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.124:44157: Network error: Client connection negotiation failed: client connection to 127.2.103.124:44157: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.103.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.124.keytab.
Entry for principal kudu/127.2.103.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.124.keytab.
I20260811 14:40:08.297825 2461 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmCSHXK/build/release/bin/kudu
/tmp/dist-test-taskmCSHXK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.124:44157
--webserver_interface=127.2.103.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.124.keytab
--principal=kudu/127.2.103.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.103.84:46613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.103.126:46519,127.2.103.125:42965,127.2.103.124:44157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:40:08.321605 7461 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } attempt: 3
Aug 11 14:40:08 dist-test-slave-twwt krb5kdc[7338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459208, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for kudu/127.2.103.125@KRBTEST.COM
I20260811 14:40:08.329516 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.321989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:42965 (local address 127.0.0.1:42452)
0811 14:40:08.322054 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.322066 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:40:08.322134 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:40:08.322301 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:40:08.322303 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:40:08.322511 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:40:08.322595 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:08.322597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.323320 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.323321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:08.324175 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:08.324177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.324585 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.324587 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:08.324671 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.325131 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:40:08.325137 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:40:08.326794 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:40:08.328535 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:08.328536 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:08.328536 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:08.328718 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:08.328929 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:08.328929 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:08.328930 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:08.328986 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:08.329254 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:40:08.329255 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:40:08.329296 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 14:40:08.329360 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 14:40:08.329441 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 14:40:08.329587 7582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.322068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42452 (local address 127.2.103.125:42965)
0811 14:40:08.322136 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.322138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.322149 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:40:08.322187 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.322188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.322202 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.322263 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.322728 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.323235 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.324271 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.324500 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.324560 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.326890 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:08.326892 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:08.326892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:08.326901 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:08.328446 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.328793 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.328794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.328794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.328860 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.329047 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.329048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.329048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.329184 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:08.329253 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:08.329405 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 14:40:08.329491 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 14:40:08.329924 7461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:40:08.330294 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.330181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:44157 (local address 127.0.0.1:54576)
0811 14:40:08.330238 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.330271 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:44157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 14:40:08.330686 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.330559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:44157 (local address 127.0.0.1:54590)
0811 14:40:08.330626 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.330659 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:44157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 14:40:08.330832 7461 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.124:44157: Network error: Client connection negotiation failed: client connection to 127.2.103.124:44157: connect: Connection refused (error 111)
I20260811 14:40:08.335770 7577 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } attempt: 1
I20260811 14:40:08.336230 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.336106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:44157 (local address 127.0.0.1:54594)
0811 14:40:08.336167 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.336205 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:44157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 14:40:08.336570 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.336478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:44157 (local address 127.0.0.1:54606)
0811 14:40:08.336522 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.336545 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:44157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 14:40:08.336653 7577 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.124:44157: Network error: Client connection negotiation failed: client connection to 127.2.103.124:44157: connect: Connection refused (error 111)
I20260811 14:40:08.362008 7461 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } attempt: 1
I20260811 14:40:08.362643 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.362454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:44157 (local address 127.0.0.1:54618)
0811 14:40:08.362531 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.362584 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:44157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 14:40:08.363073 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.362963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:44157 (local address 127.0.0.1:54634)
0811 14:40:08.363017 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.363049 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:44157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 14:40:08.363205 7461 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.124:44157: Network error: Client connection negotiation failed: client connection to 127.2.103.124:44157: connect: Connection refused (error 111)
W20260811 14:40:08.390514 7588 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 14:40:08.390864 7588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:40:08.390900 7588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:40:08.390944 7588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:40:08.392145 7588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:40:08.392216 7588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:40:08.392239 7588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:40:08.392259 7588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:40:08.392282 7588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:40:08.393919 7588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.103.84:46613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.103.126:46519,127.2.103.125:42965,127.2.103.124:44157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/krb5kdc/kudu_127.2.103.124.keytab
--principal=kudu/127.2.103.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.103.124:44157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.103.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:40:08.394243 7588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:40:08.394510 7588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:40:08.399748 7598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:40:08.399839 7603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:40:08.399904 7600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:40:08.400120 7588 server_base.cc:1034] running on GCE node
I20260811 14:40:08.400374 7588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:40:08.400652 7588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:40:08.401863 7588 hybrid_clock.cc:648] HybridClock initialized: now 1786459208401844 us; error 32 us; skew 500 ppm
Aug 11 14:40:08 dist-test-slave-twwt krb5kdc[7338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459208, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:40:08.404839 7588 init.cc:377] Logged in from keytab as kudu/127.2.103.124@KRBTEST.COM (short username kudu)
I20260811 14:40:08.404793 7577 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } attempt: 2
I20260811 14:40:08.405283 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.405108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:44157 (local address 127.0.0.1:54646)
0811 14:40:08.405188 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.405240 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:44157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 14:40:08.405472 7588 webserver.cc:460] Webserver started at http://127.2.103.124:36187/ using document root <none> and password file <none>
I20260811 14:40:08.405730 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.405621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:44157 (local address 127.0.0.1:54648)
0811 14:40:08.405673 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.405703 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.103.124:44157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 14:40:08.405756 7588 fs_manager.cc:359] Metadata directory not provided
I20260811 14:40:08.405826 7588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260811 14:40:08.405859 7577 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.103.124:44157: Network error: Client connection negotiation failed: client connection to 127.2.103.124:44157: connect: Connection refused (error 111)
I20260811 14:40:08.405989 7588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:40:08.407255 7588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-2/data/instance:
uuid: "21b217c2587a49dd9374eafc0f7025d4"
format_stamp: "Formatted at 2026-08-11 14:40:08 on dist-test-slave-twwt"
server_key: "8ddcd3dd6d47744c32f9be65ff9d55ae"
server_key_iv: "7319b50c0bd4d8e4c269c4848362fd58"
server_key_version: "encryptionkey@0"
I20260811 14:40:08.407655 7588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-2/wal/instance:
uuid: "21b217c2587a49dd9374eafc0f7025d4"
format_stamp: "Formatted at 2026-08-11 14:40:08 on dist-test-slave-twwt"
server_key: "8ddcd3dd6d47744c32f9be65ff9d55ae"
server_key_iv: "7319b50c0bd4d8e4c269c4848362fd58"
server_key_version: "encryptionkey@0"
I20260811 14:40:08.409132 7588 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:40:08.410032 7619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:40:08.410306 7588 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:40:08.410435 7588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-2/data,/tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-2/wal
uuid: "21b217c2587a49dd9374eafc0f7025d4"
format_stamp: "Formatted at 2026-08-11 14:40:08 on dist-test-slave-twwt"
server_key: "8ddcd3dd6d47744c32f9be65ff9d55ae"
server_key_iv: "7319b50c0bd4d8e4c269c4848362fd58"
server_key_version: "encryptionkey@0"
I20260811 14:40:08.410517 7588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:40:08.427278 7588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:40:08.429414 7588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:40:08.429615 7588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:40:08.433871 7588 rpc_server.cc:307] RPC server started. Bound to: 127.2.103.124:44157
I20260811 14:40:08.433912 7697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.103.124:44157 every 8 connection(s)
I20260811 14:40:08.434312 7588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-2/data/info.pb
I20260811 14:40:08.435408 7702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:40:08.437026 7702 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:40:08.438364 7461 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } attempt: 2
Aug 11 14:40:08 dist-test-slave-twwt krb5kdc[7338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459208, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.126@KRBTEST.COM for kudu/127.2.103.124@KRBTEST.COM
Aug 11 14:40:08 dist-test-slave-twwt krb5kdc[7338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459208, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.124@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:40:08.444314 2461 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmCSHXK/build/release/bin/kudu as pid 7588
I20260811 14:40:08.444454 2461 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmCSHXK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458919184199-2461-0/minicluster-data/master-2/wal/instance
I20260811 14:40:08.444695 2461 external_mini_cluster.cc:1428] Setting key a7f6f9f7476d5e6618d3944fd5b77f84
I20260811 14:40:08.448040 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.438707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:44157 (local address 127.0.0.1:54652)
0811 14:40:08.438758 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.438769 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:40:08.438809 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:40:08.439099 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:40:08.439100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:40:08.439274 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:40:08.439327 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:08.439329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.440419 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.440421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:08.441209 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:08.441211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.441897 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.441898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:08.441950 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.442525 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:40:08.442530 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:40:08.444037 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:40:08.446442 (+ 2405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:08.446443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:08.446444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:08.446772 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:08.447067 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:08.447068 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:08.447068 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:08.447138 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:08.447623 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:40:08.447624 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:40:08.447675 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 14:40:08.447822 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 14:40:08.447929 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 14:40:08.448383 7704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.437797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.126:46519 (local address 127.0.0.1:58904)
0811 14:40:08.437993 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.438030 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 14:40:08.438553 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:40:08.438991 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:40:08.438995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:40:08.439366 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:40:08.439537 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:08.439543 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.440172 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.440174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:08.441322 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:08.441325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.441783 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.441784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:08.441916 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.442520 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:40:08.442531 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:40:08.444742 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:40:08.447054 (+ 2312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:08.447061 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:08.447064 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:08.447307 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:08.447521 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:08.447521 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:08.447522 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:08.447592 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:08.447899 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:40:08.447907 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:40:08.447969 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 14:40:08.448053 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 14:40:08.448163 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":85,"threads_started":1}
I20260811 14:40:08.448635 7705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.438754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54652 (local address 127.2.103.124:44157)
0811 14:40:08.438934 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.438938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.438954 (+ 16us) server_negotiation.cc:407] Connection header received
0811 14:40:08.438986 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.438987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.439005 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.439078 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.439416 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.440344 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.441297 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.441832 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.441914 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.444150 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:08.444159 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:08.444159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:08.444167 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:08.446338 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.446867 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.446871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.446871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.446987 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.447206 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.447207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.447208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.447541 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:08.447646 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:08.447904 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 14:40:08.448034 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":114,"threads_started":1}
I20260811 14:40:08.448715 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.437813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58904 (local address 127.2.103.126:46519)
0811 14:40:08.437889 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.437891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.438121 (+ 230us) server_negotiation.cc:407] Connection header received
0811 14:40:08.438697 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.438699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.438714 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.438774 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.439626 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.440084 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.441404 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.441677 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.441756 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.444876 (+ 3120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:08.444878 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:08.444878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:08.444886 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:08.446972 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.447389 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.447389 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.447390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.447445 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.447674 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.447676 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.447676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.447809 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:08.447872 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:08.448549 (+ 677us) server_negotiation.cc:299] Negotiation successful
0811 14:40:08.448637 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 14:40:08.449446 7702 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 14:40:08 dist-test-slave-twwt krb5kdc[7338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.124@KRBTEST.COM
I20260811 14:40:08.451344 7461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5: Bootstrap starting.
I20260811 14:40:08.452190 7461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5: Neither blocks nor log segments found. Creating new log.
I20260811 14:40:08.452524 7461 log.cc:824] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5: Log is configured to *not* fsync() on all Append() calls
I20260811 14:40:08.453537 7461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5: No bootstrap required, opened a new log
I20260811 14:40:08.455302 7711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.445101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54658 (local address 127.2.103.124:44157)
0811 14:40:08.445280 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.445282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.445315 (+ 33us) server_negotiation.cc:407] Connection header received
0811 14:40:08.445693 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.445694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.445707 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.445794 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.446639 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.447351 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.448430 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.448769 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.448850 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.451700 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:08.451702 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:08.451703 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:08.451740 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:08.453991 (+ 2251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.454375 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.454376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.454377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.454459 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.454696 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.454697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.454697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.454841 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:08.455000 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:08.455081 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 14:40:08.455174 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":84,"threads_started":1}
I20260811 14:40:08.455960 7461 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } } peers { permanent_uuid: "4e818811921b444c800c9df37916259d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } } peers { permanent_uuid: "21b217c2587a49dd9374eafc0f7025d4" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } }
I20260811 14:40:08.456144 7461 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7b31f7683b74e2aa466f2d9f0dd15c5, State: Initialized, Role: FOLLOWER
I20260811 14:40:08.456444 7461 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } } peers { permanent_uuid: "4e818811921b444c800c9df37916259d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } } peers { permanent_uuid: "21b217c2587a49dd9374eafc0f7025d4" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } }
I20260811 14:40:08.456772 7713 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } } peers { permanent_uuid: "4e818811921b444c800c9df37916259d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } } peers { permanent_uuid: "21b217c2587a49dd9374eafc0f7025d4" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } } }
I20260811 14:40:08.456945 7713 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [sys.catalog]: This master's current role is: FOLLOWER
Aug 11 14:40:08 dist-test-slave-twwt krb5kdc[7338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459208, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.124@KRBTEST.COM for kudu/127.2.103.125@KRBTEST.COM
I20260811 14:40:08.456858 7461 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 14:40:08.459620 7730 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:40:08.459681 7730 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:40:08.460711 7704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.449751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.125:42965 (local address 127.0.0.1:42464)
0811 14:40:08.452937 (+ 3186us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.452950 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:40:08.453035 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:40:08.453216 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:40:08.453218 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:40:08.453372 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:40:08.453447 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:08.453449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.454324 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.454326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:08.455067 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:08.455070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.455609 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.455610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:08.455696 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.456116 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:40:08.456123 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:40:08.457605 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:40:08.459596 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:08.459597 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:08.459598 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:08.459822 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:08.460060 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:08.460061 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:08.460061 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:08.460120 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:08.460398 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:40:08.460399 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:40:08.460447 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 14:40:08.460508 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 14:40:08.460606 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3157}
I20260811 14:40:08.461081 7582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.449812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42464 (local address 127.2.103.125:42965)
0811 14:40:08.449871 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.449873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.453023 (+ 3150us) server_negotiation.cc:407] Connection header received
0811 14:40:08.453096 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.453098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.453110 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.453162 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.453547 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.454232 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.455160 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.455490 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.455552 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.457705 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:08.457706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:08.457707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:08.457715 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:08.459498 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.459928 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.459930 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.459930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.459993 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.460187 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.460188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.460188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.460311 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:08.460370 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:08.460913 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 14:40:08.460997 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
Aug 11 14:40:08 dist-test-slave-twwt krb5kdc[7338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.126@KRBTEST.COM
I20260811 14:40:08.461786 7702 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:40:08.460511 7355 master_runner.cc:186] Error getting master registration for 127.2.103.125:42965: OK, instance_id { permanent_uuid: "4e818811921b444c800c9df37916259d" instance_seqno: 1786459208258137 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 14:40:08.464109 7355 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 14:40:08.465672 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.456133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58906 (local address 127.2.103.126:46519)
0811 14:40:08.456179 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.456181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.456191 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:40:08.456212 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.456213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.456222 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.456282 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.456956 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.457520 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.458479 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.459095 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.462546 (+ 3451us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.462597 (+ 51us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:08.462597 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:08.462598 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:08.462602 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:08.464446 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.464825 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.464826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.464827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.464888 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.465115 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.465116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.465116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.465258 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:08.465403 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:08.465526 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 14:40:08.465608 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
Aug 11 14:40:08 dist-test-slave-twwt krb5kdc[7338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459208, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.124@KRBTEST.COM for kudu/127.2.103.124@KRBTEST.COM
I20260811 14:40:08.470198 7704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.462079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:44157 (local address 127.0.0.1:54674)
0811 14:40:08.462379 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.462389 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:40:08.462455 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:40:08.462662 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:40:08.462663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:40:08.462814 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:40:08.462864 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:08.462866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.463735 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.463735 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:08.464516 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:08.464518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.465022 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.465023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:08.465068 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.465728 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:40:08.465734 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:40:08.467219 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:40:08.469065 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:08.469065 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:08.469066 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:08.469262 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:08.469481 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:08.469482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:08.469483 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:08.469550 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:08.469908 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:40:08.469909 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:40:08.469946 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 14:40:08.470006 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 14:40:08.470096 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289}
I20260811 14:40:08.470485 7711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.462127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54674 (local address 127.2.103.124:44157)
0811 14:40:08.462243 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.462245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.462455 (+ 210us) server_negotiation.cc:407] Connection header received
0811 14:40:08.462510 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.462511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.462524 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.462582 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.462952 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.463649 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.464606 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.464918 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.464977 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.467311 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:08.467313 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:08.467313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:08.467321 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:08.468974 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.469337 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.469338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.469339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.469403 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.469625 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.469627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.469627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.469809 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:08.469886 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:08.470328 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 14:40:08.470404 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
Aug 11 14:40:08 dist-test-slave-twwt krb5kdc[7338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.103.125@KRBTEST.COM
I20260811 14:40:08.472522 7702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4: Bootstrap starting.
I20260811 14:40:08.473395 7702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4: Neither blocks nor log segments found. Creating new log.
I20260811 14:40:08.473783 7702 log.cc:824] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4: Log is configured to *not* fsync() on all Append() calls
I20260811 14:40:08.474560 7582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.466385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42472 (local address 127.2.103.125:42965)
0811 14:40:08.466435 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.466437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.466445 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:40:08.466469 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.466470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.466479 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.466539 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.467195 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.467848 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.468687 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.468905 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.468984 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.471485 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:08.471486 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:08.471487 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:08.471494 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:08.473275 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.473710 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.473711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.473712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.473773 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.474003 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.474005 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.474005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.474144 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:08.474340 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:08.474381 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 14:40:08.474461 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:40:08.474874 7702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4: No bootstrap required, opened a new log
I20260811 14:40:08.476794 7702 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } } peers { permanent_uuid: "4e818811921b444c800c9df37916259d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } } peers { permanent_uuid: "21b217c2587a49dd9374eafc0f7025d4" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } }
I20260811 14:40:08.476929 7702 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21b217c2587a49dd9374eafc0f7025d4, State: Initialized, Role: FOLLOWER
I20260811 14:40:08.477022 7702 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } } peers { permanent_uuid: "4e818811921b444c800c9df37916259d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } } peers { permanent_uuid: "21b217c2587a49dd9374eafc0f7025d4" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } }
I20260811 14:40:08.477385 7702 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:40:08.477350 7734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } } peers { permanent_uuid: "4e818811921b444c800c9df37916259d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } } peers { permanent_uuid: "21b217c2587a49dd9374eafc0f7025d4" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } } }
I20260811 14:40:08.477475 7734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 14:40:08.479725 7753 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:40:08.479838 7753 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:40:08.480723 2461 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:40:08.480978 7588 master_runner.cc:186] Error getting master registration for 127.2.103.125:42965: OK, instance_id { permanent_uuid: "4e818811921b444c800c9df37916259d" instance_seqno: 1786459208258137 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 14:40:08.483359 7588 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 14:40:08.483848 7422 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58906
I20260811 14:40:08.483999 7422 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 14:40:08.484055 7422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:40:08.485054 7422 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } } peers { permanent_uuid: "4e818811921b444c800c9df37916259d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } } peers { permanent_uuid: "21b217c2587a49dd9374eafc0f7025d4" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } }
I20260811 14:40:08.485298 7422 leader_election.cc:288] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4e818811921b444c800c9df37916259d (127.2.103.125:42965), 21b217c2587a49dd9374eafc0f7025d4 (127.2.103.124:44157)
I20260811 14:40:08.485766 7659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "21b217c2587a49dd9374eafc0f7025d4"
I20260811 14:40:08.485893 7659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:40:08.486831 7659 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7b31f7683b74e2aa466f2d9f0dd15c5 in term 1.
I20260811 14:40:08.487054 7380 leader_election.cc:302] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21b217c2587a49dd9374eafc0f7025d4, d7b31f7683b74e2aa466f2d9f0dd15c5; no voters:
I20260811 14:40:08.487221 7713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:40:08.487391 7713 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [term 1 LEADER]: Becoming Leader. State: Replica: d7b31f7683b74e2aa466f2d9f0dd15c5, State: Running, Role: LEADER
I20260811 14:40:08.487493 7713 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } } peers { permanent_uuid: "4e818811921b444c800c9df37916259d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } } peers { permanent_uuid: "21b217c2587a49dd9374eafc0f7025d4" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } }
I20260811 14:40:08.487963 7713 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7b31f7683b74e2aa466f2d9f0dd15c5. Latest consensus state: current_term: 1 leader_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } } peers { permanent_uuid: "4e818811921b444c800c9df37916259d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } } peers { permanent_uuid: "21b217c2587a49dd9374eafc0f7025d4" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } } }
I20260811 14:40:08.488039 7713 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:40:08.489564 7758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:40:08.489771 7758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:40:08.492087 7659 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4 [term 1 FOLLOWER]: Refusing update from remote peer d7b31f7683b74e2aa466f2d9f0dd15c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:40:08.492944 7756 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21b217c2587a49dd9374eafc0f7025d4" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:40:08.494326 7734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7b31f7683b74e2aa466f2d9f0dd15c5. Latest consensus state: current_term: 1 leader_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } } peers { permanent_uuid: "4e818811921b444c800c9df37916259d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } } peers { permanent_uuid: "21b217c2587a49dd9374eafc0f7025d4" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } } }
I20260811 14:40:08.494447 7734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:40:08.496866 7713 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } } peers { permanent_uuid: "4e818811921b444c800c9df37916259d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } } peers { permanent_uuid: "21b217c2587a49dd9374eafc0f7025d4" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } } }
I20260811 14:40:08.496976 7713 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:40:08.497026 7577 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } attempt: 3
I20260811 14:40:08.497164 7734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } } peers { permanent_uuid: "4e818811921b444c800c9df37916259d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } } peers { permanent_uuid: "21b217c2587a49dd9374eafc0f7025d4" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } } }
I20260811 14:40:08.497259 7734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:40:08.497520 7770 mvcc.cc:205] Tried to move back new op lower bound from 7317336917979934720 to 7317336917965549568. Current Snapshot: MvccSnapshot[applied={T|T < 7317336917979934720}]
I20260811 14:40:08.498211 7773 mvcc.cc:205] Tried to move back new op lower bound from 7317336917979934720 to 7317336917965549568. Current Snapshot: MvccSnapshot[applied={T|T < 7317336917979934720}]
I20260811 14:40:08.500697 7758 catalog_manager.cc:1347] Generated new cluster ID: 2fa4a66022d840f7978d8d33972f96ff
I20260811 14:40:08.500753 7758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:40:08 dist-test-slave-twwt krb5kdc[7338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786459208, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.103.125@KRBTEST.COM for kudu/127.2.103.124@KRBTEST.COM
I20260811 14:40:08.506042 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.497286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.103.124:44157 (local address 127.0.0.1:54692)
0811 14:40:08.497363 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0811 14:40:08.497376 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:40:08.497412 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:40:08.497591 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:40:08.497592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:40:08.497777 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:40:08.497816 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:08.497818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.499301 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.499302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:08.500121 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:40:08.500124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.500728 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.500729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:40:08.500788 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.501153 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:40:08.501159 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:40:08.502713 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:40:08.504840 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:08.504841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:08.504841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:08.505052 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:08.505288 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:40:08.505289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:40:08.505289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:40:08.505350 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:40:08.505618 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:40:08.505619 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:40:08.505684 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 14:40:08.505752 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 14:40:08.505870 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 14:40:08.506680 7705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.497357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54692 (local address 127.2.103.124:44157)
0811 14:40:08.497390 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.497392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.497402 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:40:08.497503 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.497505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.497517 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.497568 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.498466 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.499215 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.500222 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.500596 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.500666 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.502807 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:08.502809 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:08.502809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:08.502817 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:08.504755 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.505152 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.505154 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.505154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.505231 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.505421 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.505422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.505423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.505562 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:08.505626 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:08.506458 (+ 832us) server_negotiation.cc:299] Negotiation successful
0811 14:40:08.506547 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 14:40:08.508941 7577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d: Bootstrap starting.
I20260811 14:40:08.509783 7577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d: Neither blocks nor log segments found. Creating new log.
I20260811 14:40:08.510116 7577 log.cc:824] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d: Log is configured to *not* fsync() on all Append() calls
I20260811 14:40:08.511709 7577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d: No bootstrap required, opened a new log
I20260811 14:40:08.513938 7577 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } } peers { permanent_uuid: "4e818811921b444c800c9df37916259d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } } peers { permanent_uuid: "21b217c2587a49dd9374eafc0f7025d4" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } }
I20260811 14:40:08.514096 7577 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e818811921b444c800c9df37916259d, State: Initialized, Role: FOLLOWER
I20260811 14:40:08.514189 7577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } } peers { permanent_uuid: "4e818811921b444c800c9df37916259d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } } peers { permanent_uuid: "21b217c2587a49dd9374eafc0f7025d4" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } }
I20260811 14:40:08.514582 7577 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:40:08.514662 7541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4e818811921b444c800c9df37916259d"
I20260811 14:40:08.514730 7539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:40:08.516096 7539 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d [term 1 FOLLOWER]: Refusing update from remote peer d7b31f7683b74e2aa466f2d9f0dd15c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:40:08.517310 7541 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7b31f7683b74e2aa466f2d9f0dd15c5 in term 1.
I20260811 14:40:08.514573 7782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } } peers { permanent_uuid: "4e818811921b444c800c9df37916259d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } } peers { permanent_uuid: "21b217c2587a49dd9374eafc0f7025d4" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } } }
I20260811 14:40:08.518013 7782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:40:08.518878 7582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.489468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42478 (local address 127.2.103.125:42965)
0811 14:40:08.489510 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.489512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.491183 (+ 1671us) server_negotiation.cc:407] Connection header received
0811 14:40:08.491277 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.491279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.491290 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.491320 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.492911 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.493663 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.495550 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.496469 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.496547 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.499346 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:08.499348 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:08.499348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:08.499356 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:08.501651 (+ 2295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.517149 (+ 15498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.517151 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.517152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.517217 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.517937 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.517938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.517939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.518082 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:08.518164 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:08.518682 (+ 518us) server_negotiation.cc:299] Negotiation successful
0811 14:40:08.518786 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:40:08.519840 7711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.490709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54680 (local address 127.2.103.124:44157)
0811 14:40:08.490750 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.490751 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.498775 (+ 8024us) server_negotiation.cc:407] Connection header received
0811 14:40:08.498879 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.498880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.498890 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.498952 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.499793 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.500481 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.506812 (+ 6331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.507143 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.507184 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.514952 (+ 7768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:08.514954 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:08.514955 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:08.514963 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:08.516695 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.517586 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.517587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.517588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.517660 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.518350 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.518351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.518352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.518490 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:08.518576 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:08.519642 (+ 1066us) server_negotiation.cc:299] Negotiation successful
0811 14:40:08.519747 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 14:40:08.521366 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:08.489174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58922 (local address 127.2.103.126:46519)
0811 14:40:08.489223 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.489226 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.491025 (+ 1799us) server_negotiation.cc:407] Connection header received
0811 14:40:08.491104 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.491105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.491118 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.491186 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.497647 (+ 6461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.498368 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.503672 (+ 5304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:08.504004 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:08.504049 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:08.506481 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:40:08.506483 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:40:08.506483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:40:08.506492 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:40:08.511208 (+ 4716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.520452 (+ 9244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.520453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.520454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.520526 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:40:08.520749 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:40:08.520750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:40:08.520751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:40:08.520917 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:40:08.521032 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:08.521182 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 14:40:08.521274 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 14:40:08.518522 7777 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e818811921b444c800c9df37916259d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 14:40:08.524319 7805 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:40:08.524433 7805 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:40:08.524802 7758 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:40:08.525357 7811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7b31f7683b74e2aa466f2d9f0dd15c5. Latest consensus state: current_term: 1 leader_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } } peers { permanent_uuid: "4e818811921b444c800c9df37916259d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } } peers { permanent_uuid: "21b217c2587a49dd9374eafc0f7025d4" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } } }
I20260811 14:40:08.525466 7811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:40:08.525460 7758 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:40:08.525370 7782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } } peers { permanent_uuid: "4e818811921b444c800c9df37916259d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } } peers { permanent_uuid: "21b217c2587a49dd9374eafc0f7025d4" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } } }
I20260811 14:40:08.525560 7782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:40:08.526135 7756 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b31f7683b74e2aa466f2d9f0dd15c5" member_type: VOTER last_known_addr { host: "127.2.103.126" port: 46519 } } peers { permanent_uuid: "4e818811921b444c800c9df37916259d" member_type: VOTER last_known_addr { host: "127.2.103.125" port: 42965 } } peers { permanent_uuid: "21b217c2587a49dd9374eafc0f7025d4" member_type: VOTER last_known_addr { host: "127.2.103.124" port: 44157 } } }
I20260811 14:40:08.526245 7756 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:40:08.534811 7809 mvcc.cc:205] Tried to move back new op lower bound from 7317336917979934720 to 7317336917965549568. Current Snapshot: MvccSnapshot[applied={T|T < 7317336917979934720}]
I20260811 14:40:08.540480 7758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7b31f7683b74e2aa466f2d9f0dd15c5: Generated new TSK 0
I20260811 14:40:08.540697 7758 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 14:40:08.572074 7464 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:08.570214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58924 (local address 127.2.103.126:46519)
0811 14:40:08.570263 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.570265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.571329 (+ 1064us) server_negotiation.cc:407] Connection header received
0811 14:40:08.571443 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.571444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.571458 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.571490 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.571995 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58924: BlockingRecv error: recv got EOF from 127.0.0.1:58924 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260811 14:40:08.573161 7582 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:08.570663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42492 (local address 127.2.103.125:42965)
0811 14:40:08.570708 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.570710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.572506 (+ 1796us) server_negotiation.cc:407] Connection header received
0811 14:40:08.572615 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.572616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.572628 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.572658 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.573077 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42492: BlockingRecv error: recv got EOF from 127.0.0.1:42492 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260811 14:40:08.574110 7711 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:08.571032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54700 (local address 127.2.103.124:44157)
0811 14:40:08.571072 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.571074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.573545 (+ 2471us) server_negotiation.cc:407] Connection header received
0811 14:40:08.573648 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.573649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.573661 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.573690 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.574035 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54700: BlockingRecv error: recv got EOF from 127.0.0.1:54700 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260811 14:40:08.576431 7582 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:08.575592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42498 (local address 127.2.103.125:42965)
0811 14:40:08.575632 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.575633 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.575863 (+ 230us) server_negotiation.cc:407] Connection header received
0811 14:40:08.575982 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.575984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.576006 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.576037 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.576380 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42498: BlockingRecv error: recv got EOF from 127.0.0.1:42498 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260811 14:40:08.580156 7582 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:08.579105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42502 (local address 127.2.103.125:42965)
0811 14:40:08.579141 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.579143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.579413 (+ 270us) server_negotiation.cc:407] Connection header received
0811 14:40:08.579514 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.579514 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.579526 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.579556 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.580100 (+ 544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42502: BlockingRecv error: recv got EOF from 127.0.0.1:42502 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260811 14:40:08.584571 7582 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:08.583573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42508 (local address 127.2.103.125:42965)
0811 14:40:08.583611 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.583612 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.584026 (+ 414us) server_negotiation.cc:407] Connection header received
0811 14:40:08.584142 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.584143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.584155 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.584185 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.584517 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42508: BlockingRecv error: recv got EOF from 127.0.0.1:42508 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260811 14:40:08.590915 7582 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:08.590013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42512 (local address 127.2.103.125:42965)
0811 14:40:08.590052 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.590054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.590295 (+ 241us) server_negotiation.cc:407] Connection header received
0811 14:40:08.590399 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.590400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.590412 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.590443 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.590858 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42512: BlockingRecv error: recv got EOF from 127.0.0.1:42512 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260811 14:40:08.601500 7582 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:08.600459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42520 (local address 127.2.103.125:42965)
0811 14:40:08.600511 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.600514 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.600808 (+ 294us) server_negotiation.cc:407] Connection header received
0811 14:40:08.600946 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.600947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.600960 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.600994 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.601435 (+ 441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42520: BlockingRecv error: recv got EOF from 127.0.0.1:42520 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260811 14:40:08.620316 7582 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:08.619262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42532 (local address 127.2.103.125:42965)
0811 14:40:08.619312 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.619315 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.619631 (+ 316us) server_negotiation.cc:407] Connection header received
0811 14:40:08.619751 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.619752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.619765 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.619798 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.620252 (+ 454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42532: BlockingRecv error: recv got EOF from 127.0.0.1:42532 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260811 14:40:08.655062 7582 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:08.654005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42538 (local address 127.2.103.125:42965)
0811 14:40:08.654058 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.654061 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.654355 (+ 294us) server_negotiation.cc:407] Connection header received
0811 14:40:08.654471 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.654473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.654487 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.654519 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.654998 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42538: BlockingRecv error: recv got EOF from 127.0.0.1:42538 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260811 14:40:08.721570 7582 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:08.720735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42542 (local address 127.2.103.125:42965)
0811 14:40:08.720825 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.720828 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.720953 (+ 125us) server_negotiation.cc:407] Connection header received
0811 14:40:08.721044 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.721045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.721059 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.721138 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.721501 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42542: BlockingRecv error: recv got EOF from 127.0.0.1:42542 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 14:40:08.851704 7582 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:08.851026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42552 (local address 127.2.103.125:42965)
0811 14:40:08.851129 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:08.851131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:08.851184 (+ 53us) server_negotiation.cc:407] Connection header received
0811 14:40:08.851254 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:08.851255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:08.851271 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:08.851319 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:08.851652 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42552: BlockingRecv error: recv got EOF from 127.0.0.1:42552 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 14:40:09.109861 7582 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:40:09.109195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42564 (local address 127.2.103.125:42965)
0811 14:40:09.109286 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:09.109288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:09.109299 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:40:09.109329 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:09.109330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:09.109346 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:09.109414 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:40:09.109768 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42564: BlockingRecv error: recv got EOF from 127.0.0.1:42564 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 14:40:09.480511 7753 catalog_manager.cc:1259] Loaded cluster ID: 2fa4a66022d840f7978d8d33972f96ff
I20260811 14:40:09.480602 7753 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4: loading cluster ID for follower catalog manager: success
I20260811 14:40:09.481698 7753 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4: acquiring CA information for follower catalog manager: success
I20260811 14:40:09.482085 7753 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 21b217c2587a49dd9374eafc0f7025d4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 14:40:09.524988 7805 catalog_manager.cc:1259] Loaded cluster ID: 2fa4a66022d840f7978d8d33972f96ff
I20260811 14:40:09.525106 7805 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d: loading cluster ID for follower catalog manager: success
I20260811 14:40:09.525949 7805 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d: acquiring CA information for follower catalog manager: success
I20260811 14:40:09.526280 7805 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4e818811921b444c800c9df37916259d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 14:40:09.625872 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:09.623249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42566 (local address 127.2.103.125:42965)
0811 14:40:09.623408 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:09.623410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:09.623423 (+ 13us) server_negotiation.cc:407] Connection header received
0811 14:40:09.623491 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:09.623493 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:09.623550 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:09.623640 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:40:09.623787 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:09.624298 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:09.625077 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:09.625276 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:09.625335 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:09.625512 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:40:09.625670 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:40:09.625738 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:09.625775 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 14:40:09.625795 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":105,"threads_started":1}
I20260811 14:40:09.630324 7934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:09.627751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54710 (local address 127.2.103.124:44157)
0811 14:40:09.627890 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:09.627892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:09.627901 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:40:09.627982 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:09.627983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:09.628038 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:09.628135 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:40:09.628255 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:09.628755 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:09.629494 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:09.629711 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:09.629790 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:09.629947 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:40:09.630073 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:40:09.630185 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:09.630203 (+ 18us) server_negotiation.cc:299] Negotiation successful
0811 14:40:09.630227 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
I20260811 14:40:09.635210 7934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:09.632325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54724 (local address 127.2.103.124:44157)
0811 14:40:09.632398 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:09.632400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:09.632409 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:40:09.632434 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:09.632435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:09.632447 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:09.632503 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:40:09.632642 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:09.633407 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:09.634294 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:09.634553 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:09.634652 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:09.634790 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:40:09.634822 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:40:09.634878 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:09.635084 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 14:40:09.635112 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 14:40:09.635321 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:09.632387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42582 (local address 127.2.103.125:42965)
0811 14:40:09.632441 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:09.632443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:09.632451 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:40:09.632478 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:09.632479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:09.632491 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:09.632540 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:40:09.632774 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:09.633484 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:09.634490 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:09.634820 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:09.634886 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:09.635014 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:40:09.635043 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:40:09.635108 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:09.635199 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 14:40:09.635230 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 14:40:09.635924 7945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:40:09.632079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58936 (local address 127.2.103.126:46519)
0811 14:40:09.632616 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0811 14:40:09.632618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:40:09.632626 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:40:09.632675 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:40:09.632677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:40:09.632750 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:40:09.632837 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:40:09.633052 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:09.633809 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:09.634832 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:40:09.635265 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:40:09.635329 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:40:09.635531 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:40:09.635648 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:40:09.635721 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 14:40:09.635786 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 14:40:09.635811 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":59,"threads_started":1}
I20260811 14:40:09.637574 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 7355
I20260811 14:40:09.655081 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 7476
I20260811 14:40:09.669191 2461 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmCSHXK/build/release/bin/kudu with pid 7588
2026-08-11T14:40:09Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3107 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6249 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (290488 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST